loop optimization

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
buckit
Forum Contributor
Posts: 169
Joined: Fri Jan 01, 2010 10:21 am

loop optimization

Post by buckit »

I am using SOAP client to get XML data from a SOAP server. Then parsing the XML data into an array.

here is an example of the XML data:

Code: Select all

<NewDataSet>
    <Table>
        <I>11111</I>
        <Q>741</Q> 
        <P>25.00</P> 
        <C>25.00</C> 
     </Table>
    <Table>
        <I>222222</I>
        <Q>876</Q> 
        <P>12.00</P> 
        <C>12.00</C> 
     </Table>
</NewDataSet>
The actual XML data has 27,000 <Table> elements.

Here is the code:

Code: Select all

//SOAP it up
$param = array('this' => ****, 'that' => ****, 'other' => ****, 'stuff' => '');
$server = "http://www.someurl.com/updates.asmx?WSDL";
$client = new SoapClient($server);
$result = $client->Updater($param);
$xml = $result->UpdaterResult;
$xml = $xml->any;

//Put the XML into a usable array
$doc = new DOMDocument();
if(!$doc->loadXML($xml)){echo "doh!";}

foreach ($doc->getElementsByTagName('Table') as $node) {
	$ssi[$node->getElementsByTagName('I')->item(0)->nodeValue] = array ( 
		'q' => $node->getElementsByTagName('Q')->item(0)->nodeValue,
      	        'p' => $node->getElementsByTagName('P')->item(0)->nodeValue,
      	        'c' => $node->getElementsByTagName('C')->item(0)->nodeValue
      	);
}
The SOAP part takes about 8 seconds to complete. The entire script all together takes 235 seconds (give or take a couple seconds). the foreach is obviously the bottle neck... any recommendations on speeding it up?
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: loop optimization

Post by Christopher »

Maybe use SimpleXML or another parser instead of DomDocument.
(#10850)
buckit
Forum Contributor
Posts: 169
Joined: Fri Jan 01, 2010 10:21 am

Re: loop optimization

Post by buckit »

Going with SimpleXML and switching to a FOR loop shaved off about 10 seconds... I'll take that, but would like more.

thanks for the advice!
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: loop optimization

Post by Christopher »

If the data is that structured and simple ... maybe you could just load the file in whole, or in large chunks. You could just do some simple string ops to extract the four values for each Table entry.
(#10850)
buckit
Forum Contributor
Posts: 169
Joined: Fri Jan 01, 2010 10:21 am

Re: loop optimization

Post by buckit »

Thanks Christopher, can you give me more of an example of what you're talking about? My brain is fried today and im not following what you mean. :)



Here is some more detail.

I have a database of products from a specific distributor. The distributor offers an API to get upto date information via SOAP. They update their inventory every 20 minutes.

I need to update the quantity in my database to the current quantity from the distributor.

to do this I am putting the data from the distributor into an array like so: $current_inventory['itemNumber']['qty']

now all I need to do is loop through my database and check the item number to the array element.

so in all I would be looping the XML data to put into an array and then loop the database to update the database.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Re: loop optimization

Post by Christopher »

I was thinking that because the data is so structured you could do somthing like this:

Code: Select all

// data:         <I>11111</I>
$index = substr($line, 10, 1);
$value = substr($line, 12, strlen($line)-16);
For updating the database, do you only want to update records that have changed. If so, maybe build a SELECT to find the records that have changed and then build a single INSERT from that. That is only two queries -- rather than looping.

Also, if there are lots of updated records always then you may want to write tab delimited data to a file (fast) and then do LOAD DATA INFILE (also fast).
(#10850)
buckit
Forum Contributor
Posts: 169
Joined: Fri Jan 01, 2010 10:21 am

Re: loop optimization

Post by buckit »

Thanks man! you gave me some good ideas! I have a lot of things to try. I appreciate your help/ideas
Post Reply