PHP web crawler
Moderator: General Moderators
-
Joeiscoolone
- Forum Newbie
- Posts: 12
- Joined: Wed May 24, 2006 5:01 pm
PHP web crawler
I know that you can build a PHP search engine that will crawl the web and can rank pages by keywords but how far can a PHP search engine go can it look at how many pages link to a page and rank them like that? And how many pages could a PHP search engine be designed to index?
-
Joeiscoolone
- Forum Newbie
- Posts: 12
- Joined: Wed May 24, 2006 5:01 pm
Thanks
Thank you for the reply I appreciacte it.
How is Perl more scalable then PHP? Sure, there may be a performance difference in favor of Perl, but is that difference alone enough to warrant the comment made? Or wouldn't the true scalability of this utility be based on the solution and it's algorithms?onion2k wrote:It's possible with PHP, but it's a very silly idea. It wouldn't scale well at all.. There are much more sensible languages for that sort of thing .. Perl for a start.
When it comes to processing text (which is basically all a search engine spider does) Perl is waaaay better than PHP (in my experience).BDKR wrote:How is Perl more scalable then PHP? Sure, there may be a performance difference in favor of Perl, but is that difference alone enough to warrant the comment made? Or wouldn't the true scalability of this utility be based on the solution and it's algorithms?
That's certainly true there. Larry paid a lot of attention to that area in the earliest iterations of Perl.onion2k wrote:When it comes to processing text (which is basically all a search engine spider does) Perl is waaaay better than PHP (in my experience).BDKR wrote:How is Perl more scalable then PHP? Sure, there may be a performance difference in favor of Perl, but is that difference alone enough to warrant the comment made? Or wouldn't the true scalability of this utility be based on the solution and it's algorithms?