Bot behaviour

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
anon404
Forum Newbie
Posts: 12
Joined: Wed Dec 23, 2009 4:09 am

Bot behaviour

Post by anon404 »

I am writing a script that extracts movie plots from imdb.com, the problem is i am getting denied access. Is there a way around this?
Weirdan: Source code removed
User avatar
AbraCadaver
DevNet Master
Posts: 2572
Joined: Mon Feb 24, 2003 10:12 am
Location: The Republic of Texas
Contact:

Re: Bot behaviour

Post by AbraCadaver »

Some sites restrict access if it appears to be a bot.
Weirdan: advice removed
mysql_function(): WARNING: This extension is deprecated as of PHP 5.5.0, and will be removed in the future. Instead, the MySQLi or PDO_MySQLextension should be used. See also MySQL: choosing an API guide and related FAQ for more information.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Re: Bot behaviour

Post by Weirdan »

According to imdb.com terms of service you may not use any crawler unless you paid a fee to them (min 15.000USD annually, details here: http://www.imdb.com/licensing/). If you did, you would be using their webservice to get the data you need, not a crawler like this.
They allow limited non-commercial use of their data in personal projects, and that data is provided via ftp (details here: http://www.imdb.com/interfaces)

Therefore this crawler clearly qualifies as a program for producing illegal copies of copyrighted content.

Source code removed, original post is locked. You may continue to discuss legal ways to use imdb content here though.

AbraCadaver, you get a verbal warning for unknowingly engaging in an activity that is against forum rules.
Post Reply