Spider Type Script in PHP

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
zeeshan4it
Forum Newbie
Posts: 1
Joined: Sat Aug 20, 2005 5:28 am

Spider Type Script in PHP

Post by zeeshan4it »

Hello!

I want to add spider type script on my website to read prices from different vendors website from their product page.

Is this possible in PHP or Perl? If yes whats the script or how can i do that?

Regards,

Zeeshan
User avatar
m3mn0n
PHP Evangelist
Posts: 3548
Joined: Tue Aug 13, 2002 3:35 pm
Location: Calgary, Canada

Post by m3mn0n »

Yup.

file_get_contents() will grab the html source of a particular page. Then you can parse it however you like.

Though I highly recommend looking for an RSS or text feed or joinign some sort of API program the site may offer because this is not exactly the best way to do the job. One layout change on any of the sites and that'll screw up your entire script functionality. Whereas a data source would be a reliable and steady solution which you know won't fail.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

At a basic level file_get_contents() included in some recursive function could spider around websites BUT it doesn't work for more secure sites. For more secure sites you will need to use the cURL library ideally or fall-down to using fsockopen()...

For examples and function reference visit http://www.php.net/
User avatar
raghavan20
DevNet Resident
Posts: 1451
Joined: Sat Jun 11, 2005 6:57 am
Location: London, UK
Contact:

Post by raghavan20 »

I think it would be good to use a rss link as you have to search only one field like 'description' or 'price' of the parent tag 'item' which is far less simpler than parsing an ugly html page :)
User avatar
John Cartwright
Site Admin
Posts: 11470
Joined: Tue Dec 23, 2003 2:10 am
Location: Toronto
Contact:

Post by John Cartwright »

And you have permission from these sites to use their information?
Post Reply