We have a site that we are developing, so we have a robots.txt file on it until we are happy it doesn't conflict with rankings of our others.
However, it doesn't read it. We see it in FTP and Plesk Filemanager, but if we go to the site ...co.uk/robots.txt, it says it cannot be found.
If I rename a file to robot.txt, that works.
Because of this, Google has cached the bloody site!!
Any ideas why a robots.txt would not work. It's a brand new file, so it not corrupted either.
Why is our robots.txt file being ignored by the server?
Moderator: General Moderators
-
simonmlewis
- DevNet Master
- Posts: 4435
- Joined: Wed Oct 08, 2008 3:39 pm
- Location: United Kingdom
- Contact:
Why is our robots.txt file being ignored by the server?
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
All the best from the United Kingdom.
Re: Why is our robots.txt file being ignored by the server?
From your .htaccess post:
[text]RewriteRule ^robots.txt$ robots.php [QSA][/text]
[text]RewriteRule ^robots.txt$ robots.php [QSA][/text]
-
simonmlewis
- DevNet Master
- Posts: 4435
- Joined: Wed Oct 08, 2008 3:39 pm
- Location: United Kingdom
- Contact:
Re: Why is our robots.txt file being ignored by the server?
But we aren't using robots.php?
Do I need to create a PHP version to make that work - and if so, how?
Do I need to create a PHP version to make that work - and if so, how?
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
All the best from the United Kingdom.
Re: Why is our robots.txt file being ignored by the server?
But do you have that rule in your .htaccess? If so, there's the source of your error; it's trying to rewrite your robots.txt to a PHP file that doesn't exist.
-
simonmlewis
- DevNet Master
- Posts: 4435
- Joined: Wed Oct 08, 2008 3:39 pm
- Location: United Kingdom
- Contact:
Re: Why is our robots.txt file being ignored by the server?
Bingo. Thanks.
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
All the best from the United Kingdom.