Page 1 of 1

loop optimization

Posted: Tue Oct 05, 2010 8:51 pm
by buckit
I am using SOAP client to get XML data from a SOAP server. Then parsing the XML data into an array.

here is an example of the XML data:

Code: Select all

<NewDataSet>
    <Table>
        <I>11111</I>
        <Q>741</Q> 
        <P>25.00</P> 
        <C>25.00</C> 
     </Table>
    <Table>
        <I>222222</I>
        <Q>876</Q> 
        <P>12.00</P> 
        <C>12.00</C> 
     </Table>
</NewDataSet>
The actual XML data has 27,000 <Table> elements.

Here is the code:

Code: Select all

//SOAP it up
$param = array('this' => ****, 'that' => ****, 'other' => ****, 'stuff' => '');
$server = "http://www.someurl.com/updates.asmx?WSDL";
$client = new SoapClient($server);
$result = $client->Updater($param);
$xml = $result->UpdaterResult;
$xml = $xml->any;

//Put the XML into a usable array
$doc = new DOMDocument();
if(!$doc->loadXML($xml)){echo "doh!";}

foreach ($doc->getElementsByTagName('Table') as $node) {
	$ssi[$node->getElementsByTagName('I')->item(0)->nodeValue] = array ( 
		'q' => $node->getElementsByTagName('Q')->item(0)->nodeValue,
      	        'p' => $node->getElementsByTagName('P')->item(0)->nodeValue,
      	        'c' => $node->getElementsByTagName('C')->item(0)->nodeValue
      	);
}
The SOAP part takes about 8 seconds to complete. The entire script all together takes 235 seconds (give or take a couple seconds). the foreach is obviously the bottle neck... any recommendations on speeding it up?

Re: loop optimization

Posted: Tue Oct 05, 2010 10:48 pm
by Christopher
Maybe use SimpleXML or another parser instead of DomDocument.

Re: loop optimization

Posted: Wed Oct 06, 2010 12:44 pm
by buckit
Going with SimpleXML and switching to a FOR loop shaved off about 10 seconds... I'll take that, but would like more.

thanks for the advice!

Re: loop optimization

Posted: Wed Oct 06, 2010 1:06 pm
by Christopher
If the data is that structured and simple ... maybe you could just load the file in whole, or in large chunks. You could just do some simple string ops to extract the four values for each Table entry.

Re: loop optimization

Posted: Wed Oct 06, 2010 1:37 pm
by buckit
Thanks Christopher, can you give me more of an example of what you're talking about? My brain is fried today and im not following what you mean. :)



Here is some more detail.

I have a database of products from a specific distributor. The distributor offers an API to get upto date information via SOAP. They update their inventory every 20 minutes.

I need to update the quantity in my database to the current quantity from the distributor.

to do this I am putting the data from the distributor into an array like so: $current_inventory['itemNumber']['qty']

now all I need to do is loop through my database and check the item number to the array element.

so in all I would be looping the XML data to put into an array and then loop the database to update the database.

Re: loop optimization

Posted: Wed Oct 06, 2010 2:18 pm
by Christopher
I was thinking that because the data is so structured you could do somthing like this:

Code: Select all

// data:         <I>11111</I>
$index = substr($line, 10, 1);
$value = substr($line, 12, strlen($line)-16);
For updating the database, do you only want to update records that have changed. If so, maybe build a SELECT to find the records that have changed and then build a single INSERT from that. That is only two queries -- rather than looping.

Also, if there are lots of updated records always then you may want to write tab delimited data to a file (fast) and then do LOAD DATA INFILE (also fast).

Re: loop optimization

Posted: Wed Oct 06, 2010 2:35 pm
by buckit
Thanks man! you gave me some good ideas! I have a lot of things to try. I appreciate your help/ideas