PHP Allowed memory size problem.

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
joblafors
Forum Newbie
Posts: 1
Joined: Tue Oct 27, 2009 8:35 am

PHP Allowed memory size problem.

Post by joblafors »

Hello.
I`m writing web crawler script for one specific site, and i`m using simple_html_dom to find links and specific information in site.
But I stumbled on PHP memory problem

Code: Select all

Fatal error: Allowed memory size of 10485760 bytes exhausted (tried to allocate 16 bytes) in *****\simple_html_dom.php on line 924
.
I changed memory limit to 30MB, 50MB .. but this doesn't resolve the problem because eventually memory will be full, that is if script crawls few pages deeper in site.
Are there any suggestions how to manage memory in PHP, or are there some better html dom scripts to use (with swap files or smth like that)
Thank you !!
User avatar
infolock
DevNet Resident
Posts: 1708
Joined: Wed Sep 25, 2002 7:47 pm

Re: PHP Allowed memory size problem.

Post by infolock »

I would start doing something like this on every 20 lines of code...

Code: Select all

 
echo "<br />" . __LINE__ . "<br />";
exit;
 

I would step through and remove the exit from a line, refresh the page, remove another exit, refresh, etc until you get to the line where it's crashing.

This usually happens when you have an infinite loop, a request is failing, or a file is too big to place into memory.


More than likely this is due to a loop issue though...



Hope this helps
Post Reply