Block Bots through IP adress & number of request in time

Discussions of secure PHP coding. Security in software is important, so don't be afraid to ask. And when answering: be anal. Nitpick. No security vulnerability is too small.

Moderator: General Moderators

Post Reply
chuy244
Forum Newbie
Posts: 3
Joined: Thu Jul 03, 2008 3:20 pm

Block Bots through IP adress & number of request in time

Post by chuy244 »

Here at my company we have this search engine for our site running on a linux server and code in PHP
I've been asked to blok spider or bots using the following aproach:

Bots and spiders are web applications that scan sites automatically, therefore request made to the site occur by hundres even thousand on a short period of time and by the same agent (same IP adress).

... So, knowing this you could say it is posible to block a bot or a spider through a PHP code that identifies when an agent or an IP adress is making too many request to your server in a really short time (which you could assume it's imposible for a human browsing your page).

I know it is posible to use PHP to collect IP adresses, write them to a text file and get them later to block that IP adress if it returns. What I don't quite know how to do is how to identify an agent through a PHP script when it has been making several request to the server in a short time. Maybe someone here could help me. Thanks
User avatar
tecktalkcm0391
DevNet Resident
Posts: 1030
Joined: Fri May 26, 2006 9:25 am
Location: Florida

Re: Block Bots through IP adress & number of request in time

Post by tecktalkcm0391 »

Set up a database, and record IP's (and time of request) in it... each page needs to have a code that checks the IP against the database... and if X request come back in the last X seconds/minutes... send them a 404 error or something using header()
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: Block Bots through IP adress & number of request in time

Post by VladSun »

There are 10 types of people in this world, those who understand binary and those who don't
Post Reply