Categories and Products Picker Duplicate Content Fix

I could not find a fix for duplicate content caused by search engines crawling dispatch values categories.picker and produts.picker. I did not find anything relevent for V2.2.4, this issue was only resolved in V3.0.2.



To find out if you’ve been unfortunate, replace www.yourwebsite.com with your domain name in the following Google search:


<br />
https://www.google.co.uk/#hl=en&safe=off&tbo=d&sclient=psy-ab&q=inurl:http:%2F%2Fwww.yourwebsite.com%2Findex.php%3Fdispatch%3Dcategories.picker<br />

```<br />
```php
<br />
https://www.google.co.uk/#hl=en&safe=off&tbo=d&sclient=psy-ab&q=inurl:http:%2F%2Fwww.yourwebsite.com%2Findex.php%3Fdispatch%3Dproducts.picker<br />

```<br />
<br />
Now on to the fix, which I have now applied to both V2.2.4 and V2.2.5. It may work for previous versions of V2, but I have no development stores running any previous versions. The fix involves adding a 'nofollow' tag to the links to these pages.<br />
<br />
For products.picker duplicate content:<br />
<br />
Open<br />
```php
/skins/your_skin/customer/pickers/products_picker.tpl
```<br />
Find:<br />
```php
<br />
  {include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="products.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&aoc=`$aoc`&data_id=`$data_id`" but_text=$but_text|default:$lang.add_products but_role="add" but_rev="content_`$data_id`" but_meta="cm-dialog-opener"}<br />

```<br />
Replace with:<br />
```php
<br />
{include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="products.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&aoc=`$aoc`&data_id=`$data_id`" but_text=$but_text|default:$lang.add_products but_role="add" but_rev="content_`$data_id`" but_meta="cm-dialog-opener" but_rel="nofollow"}<br />

```<br />
<br />
And for categories.picker<br />
Open<br />
```php
/skins/your_skin/customer/pickers/categories_picker.tpl
```<br />
Find:<br />
```php
<br />
  {include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="categories.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&root=`$default_name`&except_id=`$except_id`&data_id=`$data_id``$extra_url`" but_text=$_but_text but_role=$_but_role but_rev="content_`$data_id`" but_meta="cm-dialog-opener"}<br />

```<br />
Replace with:<br />
```php
<br />
  {include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="categories.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&root=`$default_name`&except_id=`$except_id`&data_id=`$data_id``$extra_url`" but_text=$_but_text but_role=$_but_role but_rev="content_`$data_id`" but_meta="cm-dialog-opener" but_rel="nofollow"}<br />

```<br />
<br />
If you only wish to implement the fix to stop duplicate content for products.picker, categories.picker or both, you still need to add the 'but_rel' variable to the appropriate buttons.<br />
Open:<br />
```php
/skins/your_skin/customer/buttons/button.tpl
```<br />
Find:<br />
```php
<br />
<span class="button{$suffix}" {if $but_id}id="{$but_id}"{/if}><a {if $but_href}href="{$but_href|fn_url}"{/if}{if $but_onclick} onclick="{$but_onclick} return false;"{/if} {if $but_target}target="{$but_target}"{/if} class="{if $but_meta}{$but_meta} {/if}" {if $but_rev}rev="{$but_rev}"{/if}>{$but_text}</a></span><br />

```<br />
Replace with:<br />
```php
<br />
<span class="button{$suffix}" {if $but_id}id="{$but_id}"{/if}><a {if $but_href}href="{$but_href|fn_url}"{/if}{if $but_onclick} onclick="{$but_onclick} return false;"{/if} {if $but_target}target="{$but_target}"{/if} class="{if $but_meta}{$but_meta} {/if}" {if $but_rel} rel="{$but_rel}"{/if} {if $but_rev}rev="{$but_rev}"{/if}>{$but_text}</a></span><br />

```<br />
<br />
Refresh the site cache. Now all products.picker and categories.picker buttons/links have the 'nofollow' attribute. Wait for Google to re-crawl and re-love your site again.

Cheers StellarBytes



John

For the categories.picker Google turned up no results. For the products.picker it turned up about 994.



I'm on 3.0.6.



I guess I need to apply this fix…



Note that I've been using the canonical addon for years.

Yes I sent the info to CS-Cart, the fix was supposed to be applied post-3.0.4 but was never applied in 3.0.5 or 3.0.6 but has been applied in V4. If you are on CS-Cart V2 or V3, you need to apply this fix.

[quote name='StellarBytes' timestamp='1373658651' post='165272']

Yes I sent the info to CS-Cart, the fix was supposed to be applied post-3.0.4 but was never applied in 3.0.5 or 3.0.6 but has been applied in V4. If you are on CS-Cart V2 or V3, you need to apply this fix.

[/quote]



You're a lifesaver. Thanks so much for providing this!

Hmmm. I went to make this change in the products_picker.tpl file, and the nofollow is already there.



So, looks like Google is not honoring the nofollow in this case?



Can this be fixed by adding a statement to my robots.txt file or not?



As I said above, I'm using 3.0.6 and have been since about the end of February.

[quote name='kingsleypress' timestamp='1373993458' post='165420']Can this be fixed by adding a statement to my robots.txt file or not?[/quote]



Yep. If I have said it once, I have said it a million times. If you have SEO add on enabled add:


Disallow: /index.php

Not sure what's going on with this, but Google now lists 1580 URLs for the products.picker.

Hi



I've tested both urls and get no results for both. The site is on v3.0.5



I've already added Disallow: /*? to the robots.txt file, this has resulted in a huge drop in indexed urls (the site has 2700k pages and had +85k indexed urls, most relating to url parameters being indexed (currencies etc). This hasn't stopped the strange urls I am getting:



/index.php?target=gift_certificates&mode=free_products&search_products=Y&page=322



Google WMT reports 404s that are linked to from urls of this nature. Google cannot even access these.



I'm trying the suggestion by The Tool… I'll let you know



Steve

/index.php?target= hasn't been used since I think 1.3.x so the page should be returning a 404.

Steve, exactly which version of the cart are you running? I assume by the fact you have “target=” URL's that this is the 1.3.5 store as per your signature. I have never touched V1 in any shape or form, so can't advise on that one.



Setting rel=“nofollow” on links won't stop them being indexed if they are already indexed. Best to block them using .htaccess.



I actually looked at various sites and I am yet to see anyone actually buying a product for someone via gift certificates (ie. the product as a gift which results in products.picker URL's), so I removed the option to buy a product as a gift. Sorted. On the new stores I have launched where I have put this fix in place before going live, I haven't seen any products.picker nor categories.picker URL's being indexed in search engines.

Hi Stellar



The this particular site used to be 1.3.5 and moved up to v2.2.4 before going to 3.0.4 and finally 3.0.5



I've previously contacted CS Cart regarding the issue and they seemed to imply the links were still indexed. I blocked them in robots.txt a while back from accessing any urls with a ? in them and now blocked index.php from being indexed as a suggestion above.



The “target=” urls don't appear to be indexed and aren't accessible via browser yet are still showing up in Google WMTs as data under the “linked from” tab as though Google has accessed the page and found a 404 on that page.



I'm going to implement the .picker fixes today.



Cheers



Steve

Hi All



Unfortunately we're still getting the strange urls albeit less than before.



A very strange issue indeed.



Steve

Can you give further examples of the URL structures, not the full URL as this will only encourage Google to crawl and index these URL's. Does Google Webmaster Tools show these strange URL's - if so, it should also show which page(s) the URL's are linked from.

Hi Stellar



This is the 404 from google webmaster tools.

/protective-motorcycle-clothing/motorcycle-clothing-by-type/motorcycle-helmets/visors/weise-waterproof-over-gloves.html



This is the link listed under the “Linked From” tab.

/protective-motorcycle-clothing/motorcycle-clothing-by-type/motorcycle-helmets/visors/index.php?target=gift_certificates&mode=free_products&search_products=Y&page=367



The 404 link is not even correct. The url structure for products is http://www.domain.co.uk/product-name.html and this product is live and working. The category structure is correct but not for this product, this is a motorcycle helmet visor category and the product is a pair of gloves.



Google is blocked in the robots.txt file from crawling any urls with a ? in them.



Thanks for coming back. Hope this helps.



Cheers



Steve

this not working as im still getting spidered with pickers…



4.03

[quote name='vanderex' timestamp='1392924288' post='177899']

this not working as im still getting spidered with pickers…



4.03

[/quote]



any solutions?

I've recently upgrade the store mentioned above to v.4.2.4 and Google WMTs is still reporting 404s for urls including /index.php?target=gift_certificates



I've contacted CS Cart previously only to be told target isn't being used in CS Cart anymore, which I know that.



Is anyone else getting this?



Steve

Hi,

Just an update, we're on 4.3.5 and one of our website has this issue,

Any suggestions on how to fix it for this branch?

Stephen

Hi,

Just an update, we're on 4.3.5 and one of our website has this issue,

Any suggestions on how to fix it for this branch?

Stephen

It's been a while since I experienced this problem, but can happily take a look for you if you wish.