Is there a HTACCESS string that stops duplicate pages?

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

I found the answer. It's because I had QSA in there. Changing that to [L] and putting the $ in where you said, works a treat.
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
User avatar
Celauran
Moderator
Posts: 6427
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Is there a HTACCESS string that stops duplicate pages?

Post by Celauran »

Is that going to be OK for your use case? QSA will preserve things like ?page=2. Make sure you don't need that functionality.
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

if we now go to:
/a_productedit
It obviously puts a / on the end if edit.
It's posting $id thru to that page, but when the page loads it's empty. It's as though $id has not be POSTed thru to the page.

Is that because the [L] as in the rules above that part of the structure?
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

It seems that when it's a URL that is in this structured manner, if it's posting data thru a POST form, it doesn't post it through to that page.
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
User avatar
Celauran
Moderator
Posts: 6427
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Is there a HTACCESS string that stops duplicate pages?

Post by Celauran »

Your redirect is turning the POST request into a GET
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

How? We haven't altered that structure in HTACCESS....
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

If I remove this code from HTACCESS it works.

why?

Code: Select all

RewriteCond %{HTTP_HOST} !=""
RewriteCond %{THE_REQUEST} ^[A-Z]+\s//+(.*)\sHTTP/[0-9.]+$ [OR]
RewriteCond %{THE_REQUEST} ^[A-Z]+\s(.*/)/+\sHTTP/[0-9.]+$
RewriteRule .* http://%{HTTP_HOST}/%1 [R=301,L]

# Remove multiple slashes anywhere in URL
RewriteCond %{REQUEST_URI} ^(.*)//(.*)$
RewriteRule . %1/%2 [R=301,L]


RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*[^/])$ /$1/ [L,R=301]
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
User avatar
Celauran
Moderator
Posts: 6427
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Is there a HTACCESS string that stops duplicate pages?

Post by Celauran »

Because you've removed the redirects.
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

You've lost me sorry..... ??
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
User avatar
Celauran
Moderator
Posts: 6427
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Is there a HTACCESS string that stops duplicate pages?

Post by Celauran »

simonmlewis wrote:if we now go to:
/a_productedit
It obviously puts a / on the end if edit.
Where is that happening?
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

It's in HTACCESS. We have been told that a page like /big-shirts, and /big-shirts/ or /contact and /contact/ are duplicate pages. So we had to set it to only show it with / or without.

HTACCESS does that for us. But it's that which is causing this problem.

Code: Select all

DirectoryIndex index.php index.html index.htm
order allow,deny
allow from all 
Options +FollowSymLinks
Options +Indexes
RewriteEngine On
# RewriteCond %{HTTPS} !=on
# RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

# Remove multiple slashes after domain
RewriteCond %{HTTP_HOST} !=""
RewriteCond %{THE_REQUEST} ^[A-Z]+\s//+(.*)\sHTTP/[0-9.]+$ [OR]
RewriteCond %{THE_REQUEST} ^[A-Z]+\s(.*/)/+\sHTTP/[0-9.]+$
RewriteRule .* http://%{HTTP_HOST}/%1 [R=301,L]

# Remove multiple slashes anywhere in URL
RewriteCond %{REQUEST_URI} ^(.*)//(.*)$
RewriteRule . %1/%2 [R=301,L]


RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*[^/])$ /$1/ [L,R=301]

RewriteRule ^(blog)($|/) - [QSA]

# old url rewrite
RewriteRule ^categ/([0-9]+)/([^/]+)/$ /index.php?page=categ&c=$1&cname=$2 [L]
RewriteRule ^categ/page/([0-9]+)/([^/]+)/([0-9]+)/$ /index.php?page=categ&c=$1&cname=$2&pagenum=$3 [L]
RewriteRule ^subcateg/([0-9]+)/([^/]+)/([0-9]+)/([^/]+) /index.php?page=subcateg&c=$1&cname=$2&s=$3&sname=$4&menu=sub [L]
RewriteRule ^subcateg/page/([0-9]+)/([^/]+)/([0-9]+)/([^/]+)/([0-9]+) /index.php?page=subcateg&c=$1&cname=$2&s=$3&sname=$4&pagenum=$5 [L]

RewriteRule ^product/([0-9]+)/([^/]+)/([0-9]+)/([^/]+)/([0-9]+)/([^/]+)/$ /index.php?page=product&c=$1&cname=$2&s=$3&sname=$4&product=$5&h=$6 [L]
# end of old url rewrite


# NEW URLS
RewriteRule ^categ/([^/]+)/([0-9]+)/$ /index.php?page=categ&cname=$1&pagenum=$2 [L] 
RewriteRule ^categ/([^/]+)/$ /index.php?page=categ&cname=$1 [L]

RewriteRule ^subcateg/([^/]+)/([^/]+)/([0-9]+)/$ /index.php?page=subcateg&cname=$1&sname=$2&pagenum=$3 [L]
RewriteRule ^subcateg/([^/]+)/([^/]+)/$ /index.php?page=subcateg&cname=$1&sname=$2 [L]

RewriteRule ^product/([^/]+)/([^/]+)/([0-9]+)/([^/]+)/$ /index.php?page=product&cname=$1&sname=$2&product=$3&h=$4 [L]
# END OF NEW URLS



RewriteRule ^knowledge/([0-9]+) /index.php?page=knowledge&id=$1 [QSA]
RewriteRule ^knowledge/answer/([0-9]+)/([0-9]+) /index.php?page=knowledge&id=$1&id_link=$2 [QSA]
RewriteRule ^pricedrop/page/([0-9]+)/ /index.php?page=pricedrop&pagenum=$1 [QSA]
RewriteRule ^productsnew/page/([0-9]+)/ /index.php?page=productsnew&pagenum=$1 [QSA]


RewriteRule ^productzoom/([0-9]+)/([^/]+)/([0-9]+)/([^/]+)/([0-9]+)/([^/]+) /index.php?page=productzoom&c=$1&cname=$2&s=$3&sname=$4&product=$5&h=$6 [QSA]
RewriteRule ^pricematch/([0-9]+) /index.php?page=pricematch&id=$1 [QSA]

RewriteRule ^type/([^/]+) /index.php?page=type&type=$1 [QSA]
RewriteRule ^use/([^/]+) /index.php?page=use&use=$1 [QSA]

RewriteRule ^back-in-stock/page/([0-9]+)/([^/]+) /index.php?page=back-in-stock&pagenum=$1&power=$2 [L,QSA]

RewriteRule ^romancode/([^/]+) /index.php?page=romancode&romancode=$1 [QSA]
RewriteRule ^productsall/([0-9]+)/ /index.php?page=productsall/ [QSA]

RewriteRule ^productsall/page/([0-9]+)/ /index.php?page=productsall&pagenum=$1/ [QSA]
RewriteRule ^manufacturers/([^/]+) /index.php?page=manufacturers&manufacturer=$1 [QSA]
RewriteRule ^accessories-manufacturers/([^/]+) /index.php?page=accessories-manufacturers&manufacturer=$1 [QSA]
RewriteRule ^product-tags/page/([0-9]+)/([^/]+) /index.php?page=product-tags&pagenum=$1&producttag=$2 [QSA]
RewriteRule ^product-tags/([^/]+) /index.php?page=product-tags&producttag=$1 [QSA]
RewriteRule ^products-wrapped/page/([0-9]+)/ /index.php?page=products-wrapped&pagenum=$1/ [QSA]
RewriteRule ^videos/([0-9]+) /index.php?page=videos&catid=$1 [QSA]
RewriteRule ^videos/product/([0-9]+)/([0-9]+) /index.php?page=videos&catid=$1&id=$2 [QSA]
RewriteRule ^videos/product-search/([0-9]+)/([0-9]+)/([^/]+)/([^/]+) /index.php?page=videos&catid=$1&id=$2&search=$3&searchvideo=$4 [QSA]

RewriteRule ^([^/\.]+)/?$ index.php?page=$1 [L]

RewriteRule ^$ index.php?page=home [QSA]

RewriteRule ^robots.txt$ robots.php [QSA]
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
User avatar
Celauran
Moderator
Posts: 6427
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Is there a HTACCESS string that stops duplicate pages?

Post by Celauran »

simonmlewis wrote:HTACCESS does that for us.
Yes but how?
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

I believe it is this line:

Code: Select all

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*[^/])$ /$1/ [L,R=301]
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
User avatar
Celauran
Moderator
Posts: 6427
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Is there a HTACCESS string that stops duplicate pages?

Post by Celauran »

Indeed.

So, as I was saying earlier, your redirect is converting your POST request into a GET request.
simonmlewis
DevNet Master
Posts: 4435
Joined: Wed Oct 08, 2008 3:39 pm
Location: United Kingdom
Contact:

Re: Is there a HTACCESS string that stops duplicate pages?

Post by simonmlewis »

Ohhhh yes I see what you mean.
So is there no way around that?
I guess if the / are added to the Prods, Cats and Subs, it doesn't matter about the rest??
The worry is that a page can be "duplicated" by a / on the end. But yes I do see the cause now.
Love PHP. Love CSS. Love learning new tricks too.
All the best from the United Kingdom.
Post Reply