Page 1 of 1

HTML Parsing Optimization

Posted: Sun Aug 14, 2011 5:03 am
by siderp21
i am trying to create script which gets prices from various websites, the problem is that it uses a lot of memory. When i execute it RAM usage increase 200mb if i'm parsing ~30 urls.

I want that script be able to scrap ~1000 urls. I am begginer at php, i do not know what to do. Thanks for your help.

$link - parsing URL,
$info - parsing info.

Code: Select all

include('simple_html_dom.php');


function extract_numbers($string)
{
preg_match_all('/([\d]+)/', $string, $match);
return $match[0];
}

function get_price($link,$info)

{
$html = file_get_html($link);

foreach($html->find($info) as $e)

$price = extract_numbers($e->innertext);
return $price[0];
}

Code: Select all

$sql="SELECT * FROM getP_pinfo";
$result=mysql_query($sql);

while($row = mysql_fetch_assoc($result)) {
extract($row);

echo get_price($URL,$info);

}
mysql_free_result($result);
mysql_close();

Edit: Solved.