Page 1 of 1

Spider Type Script in PHP

Posted: Sat Aug 20, 2005 5:45 am
by zeeshan4it
Hello!

I want to add spider type script on my website to read prices from different vendors website from their product page.

Is this possible in PHP or Perl? If yes whats the script or how can i do that?

Regards,

Zeeshan

Posted: Sat Aug 20, 2005 6:47 am
by m3mn0n
Yup.

file_get_contents() will grab the html source of a particular page. Then you can parse it however you like.

Though I highly recommend looking for an RSS or text feed or joinign some sort of API program the site may offer because this is not exactly the best way to do the job. One layout change on any of the sites and that'll screw up your entire script functionality. Whereas a data source would be a reliable and steady solution which you know won't fail.

Posted: Sat Aug 20, 2005 12:44 pm
by Chris Corbyn
At a basic level file_get_contents() included in some recursive function could spider around websites BUT it doesn't work for more secure sites. For more secure sites you will need to use the cURL library ideally or fall-down to using fsockopen()...

For examples and function reference visit http://www.php.net/

Posted: Sat Aug 20, 2005 9:32 pm
by raghavan20
I think it would be good to use a rss link as you have to search only one field like 'description' or 'price' of the parent tag 'item' which is far less simpler than parsing an ugly html page :)

Posted: Sun Aug 21, 2005 12:15 am
by John Cartwright
And you have permission from these sites to use their information?