robots allowed to index only few files
Posted: Wed Nov 25, 2009 4:39 am
I know that if I write in robots.txt file the following lines:
all robots won`t be allowed to index the files of the site.
My question is: "How do I tell robots not to index the files of site EXCEPT the following three files:first.php, second.php, third.php?". Because, for instance, my site has 100 files and I want only three of them to be indexed by robots. It would be much easier for me to write in robots.txt file 3 lines for each file than to write 97 lines with "Disallow".
Code: Select all
User-agent: *
Disallow: /
My question is: "How do I tell robots not to index the files of site EXCEPT the following three files:first.php, second.php, third.php?". Because, for instance, my site has 100 files and I want only three of them to be indexed by robots. It would be much easier for me to write in robots.txt file 3 lines for each file than to write 97 lines with "Disallow".