Why is our robots.txt file being ignored by the server?
Posted: Tue Apr 07, 2015 9:24 am
We have a site that we are developing, so we have a robots.txt file on it until we are happy it doesn't conflict with rankings of our others.
However, it doesn't read it. We see it in FTP and Plesk Filemanager, but if we go to the site ...co.uk/robots.txt, it says it cannot be found.
If I rename a file to robot.txt, that works.
Because of this, Google has cached the bloody site!!
Any ideas why a robots.txt would not work. It's a brand new file, so it not corrupted either.
However, it doesn't read it. We see it in FTP and Plesk Filemanager, but if we go to the site ...co.uk/robots.txt, it says it cannot be found.
If I rename a file to robot.txt, that works.
Because of this, Google has cached the bloody site!!
Any ideas why a robots.txt would not work. It's a brand new file, so it not corrupted either.