I`m writing web crawler script for one specific site, and i`m using simple_html_dom to find links and specific information in site.
But I stumbled on PHP memory problem
Code: Select all
Fatal error: Allowed memory size of 10485760 bytes exhausted (tried to allocate 16 bytes) in *****\simple_html_dom.php on line 924I changed memory limit to 30MB, 50MB .. but this doesn't resolve the problem because eventually memory will be full, that is if script crawls few pages deeper in site.
Are there any suggestions how to manage memory in PHP, or are there some better html dom scripts to use (with swap files or smth like that)
Thank you !!