Page 1 of 1
PHP Web Crawler
Posted: Tue May 30, 2006 9:00 pm
by Joeiscoolone
Why is a PHP Web Crawler slower than a Web Crawler made in C++?
Posted: Tue May 30, 2006 9:04 pm
by tecktalkcm0391
Its because C++ is an less complex when it comes to coverting it to Binary.
Posted: Tue May 30, 2006 9:05 pm
by feyd
- PHP is an interpreted language.
- It has more finite limits of memory consumption along with larger amounts of memory consumption for each variable.
- It's not built for performance, but more interoperability.
Thanks
Posted: Wed May 31, 2006 12:08 am
by Joeiscoolone
Thank you again, I am very interested in a PHP web crawler.
Posted: Wed May 31, 2006 6:39 am
by MrPotatoes
oh sweet i didn't even know that C++ was even an option for this. are there any tutorials for this? i need tomake some tools for my site to crall the website application. i thought i'd have to do it either in PHP or Perl or some other language i don't know.
Re: PHP Web Crawler
Posted: Wed May 31, 2006 8:37 am
by onion2k
Joeiscoolone wrote:Why is a PHP Web Crawler slower than a Web Crawler made in C++?
The speed of any application is faaaaaar more down to the implementation than the language. A competent coder could write a fast crawler in PHP, and a poor coder would write a slow one in C++.
Merely writing your software in a different language won't instantly make it faster.
That said, some languages lend themselves to techniques that intrinsically make things quicker. C++ isn't one though .. I know for a fact that if I wrote a crawler in C++ it would be slower than one I'd write in PHP. Coz I suck at C++.
Re: PHP Web Crawler
Posted: Wed May 31, 2006 12:44 pm
by Roja
Joeiscoolone wrote:Why is a PHP Web Crawler slower than a Web Crawler made in C++?
Several reasons:
1. Overhead - Because PHP is interpreted, and run via a webserver service (generally), it has multiple layers of overhead
2. Memory limits - Web crawling requires substantial memory to parse, which is something PHP is generally limited by (8mb by default)
3. DB limits - While C can utilize the full memory of a system, PHP has only 8mb. As a result, most quasi-permanent data is stored to a database, and accessed from there - that adds I/O time and substantial overhead
4. Purpose - C is pretty much ideal for the task of large array searching, while PHP offers far less support for it.
Not to mention, the coder will make a big difference in capabilities, regardless of language.
Re: PHP Web Crawler
Posted: Thu Oct 01, 2015 1:56 am
by Rijo Mon R
how to build a price comparison application.