Page 1 of 1

PHP Web Crawler

Posted: Tue May 30, 2006 9:00 pm
by Joeiscoolone
Why is a PHP Web Crawler slower than a Web Crawler made in C++?

Posted: Tue May 30, 2006 9:04 pm
by tecktalkcm0391
Its because C++ is an less complex when it comes to coverting it to Binary.

Posted: Tue May 30, 2006 9:05 pm
by feyd
  1. PHP is an interpreted language.
  2. It has more finite limits of memory consumption along with larger amounts of memory consumption for each variable.
  3. It's not built for performance, but more interoperability.

Thanks

Posted: Wed May 31, 2006 12:08 am
by Joeiscoolone
Thank you again, I am very interested in a PHP web crawler.

Posted: Wed May 31, 2006 6:39 am
by MrPotatoes
oh sweet i didn't even know that C++ was even an option for this. are there any tutorials for this? i need tomake some tools for my site to crall the website application. i thought i'd have to do it either in PHP or Perl or some other language i don't know.

Re: PHP Web Crawler

Posted: Wed May 31, 2006 8:37 am
by onion2k
Joeiscoolone wrote:Why is a PHP Web Crawler slower than a Web Crawler made in C++?
The speed of any application is faaaaaar more down to the implementation than the language. A competent coder could write a fast crawler in PHP, and a poor coder would write a slow one in C++.

Merely writing your software in a different language won't instantly make it faster.

That said, some languages lend themselves to techniques that intrinsically make things quicker. C++ isn't one though .. I know for a fact that if I wrote a crawler in C++ it would be slower than one I'd write in PHP. Coz I suck at C++.

Re: PHP Web Crawler

Posted: Wed May 31, 2006 12:44 pm
by Roja
Joeiscoolone wrote:Why is a PHP Web Crawler slower than a Web Crawler made in C++?
Several reasons:

1. Overhead - Because PHP is interpreted, and run via a webserver service (generally), it has multiple layers of overhead

2. Memory limits - Web crawling requires substantial memory to parse, which is something PHP is generally limited by (8mb by default)

3. DB limits - While C can utilize the full memory of a system, PHP has only 8mb. As a result, most quasi-permanent data is stored to a database, and accessed from there - that adds I/O time and substantial overhead

4. Purpose - C is pretty much ideal for the task of large array searching, while PHP offers far less support for it.

Not to mention, the coder will make a big difference in capabilities, regardless of language.

Re: PHP Web Crawler

Posted: Thu Oct 01, 2015 1:56 am
by Rijo Mon R
how to build a price comparison application.