HTML Parsing Optimization

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
siderp21
Forum Newbie
Posts: 2
Joined: Sun Aug 14, 2011 4:52 am

HTML Parsing Optimization

Post by siderp21 »

i am trying to create script which gets prices from various websites, the problem is that it uses a lot of memory. When i execute it RAM usage increase 200mb if i'm parsing ~30 urls.

I want that script be able to scrap ~1000 urls. I am begginer at php, i do not know what to do. Thanks for your help.

$link - parsing URL,
$info - parsing info.

Code: Select all

include('simple_html_dom.php');


function extract_numbers($string)
{
preg_match_all('/([\d]+)/', $string, $match);
return $match[0];
}

function get_price($link,$info)

{
$html = file_get_html($link);

foreach($html->find($info) as $e)

$price = extract_numbers($e->innertext);
return $price[0];
}

Code: Select all

$sql="SELECT * FROM getP_pinfo";
$result=mysql_query($sql);

while($row = mysql_fetch_assoc($result)) {
extract($row);

echo get_price($URL,$info);

}
mysql_free_result($result);
mysql_close();

Edit: Solved.
Post Reply