Page 1 of 1

detect searchengine-bot

Posted: Mon Dec 11, 2006 4:52 am
by potato
Hi,

i'm looking for the best way to detect a searchengine-bot, for example if google or yahoo passes to crawl my site.

Anyone has some experience with this?

greetings, tom

Posted: Mon Dec 11, 2006 5:02 am
by volka
You can examine the user-agent string (e.g. Googlebot) and (maybe) the ip range.

Posted: Mon Dec 11, 2006 5:11 am
by aaronhall
The user agent is available with $_SERVER['HTTP_USER_AGENT']. A list of web spider user agents is available at http://www.psychedelix.com/agents/index.shtml or http://www.psychedelix.com/agents/index.shtml

Posted: Mon Dec 11, 2006 10:39 am
by feyd
get_browser() can often be a simple and fast way to determine if the requesting agent suggests further digging.