Jump to content

  • You cannot start a new topic
  • You cannot reply to this topic

Categories and Products Picker Duplicate Content Fix Rate Topic   - - - - -

 
  • StellarBytes
  • Senior Member
  • Members
  • Join Date: 08-Aug 11
  • 1807 posts

Posted 07 February 2013 - 05:10 PM #1

I could not find a fix for duplicate content caused by search engines crawling dispatch values categories.picker and produts.picker. I did not find anything relevent for V2.2.4, this issue was only resolved in V3.0.2.

To find out if you've been unfortunate, replace www.yourwebsite.com with your domain name in the following Google search:

https://www.google.co.uk/#hl=en&safe=off&tbo=d&sclient=psy-ab&q=inurl:http:%2F%2Fwww.yourwebsite.com%2Findex.php%3Fdispatch%3Dcategories.picker
https://www.google.co.uk/#hl=en&safe=off&tbo=d&sclient=psy-ab&q=inurl:http:%2F%2Fwww.yourwebsite.com%2Findex.php%3Fdispatch%3Dproducts.picker

Now on to the fix, which I have now applied to both V2.2.4 and V2.2.5. It may work for previous versions of V2, but I have no development stores running any previous versions. The fix involves adding a 'nofollow' tag to the links to these pages.

For products.picker duplicate content:

Open
/skins/your_skin/customer/pickers/products_picker.tpl
Find:
  {include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="products.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&aoc=`$aoc`&data_id=`$data_id`" but_text=$but_text|default:$lang.add_products but_role="add" but_rev="content_`$data_id`" but_meta="cm-dialog-opener"}
Replace with:
{include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="products.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&aoc=`$aoc`&data_id=`$data_id`" but_text=$but_text|default:$lang.add_products but_role="add" but_rev="content_`$data_id`" but_meta="cm-dialog-opener" but_rel="nofollow"}

And for categories.picker
Open
/skins/your_skin/customer/pickers/categories_picker.tpl
Find:
  {include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="categories.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&root=`$default_name`&except_id=`$except_id`&data_id=`$data_id``$extra_url`" but_text=$_but_text but_role=$_but_role but_rev="content_`$data_id`" but_meta="cm-dialog-opener"}
Replace with:
  {include file="buttons/button.tpl" but_id="opener_picker_`$data_id`" but_href="categories.picker?display=`$display`&picker_for=`$picker_for`&extra=`$extra_var`&checkbox_name=`$checkbox_name`&root=`$default_name`&except_id=`$except_id`&data_id=`$data_id``$extra_url`" but_text=$_but_text but_role=$_but_role but_rev="content_`$data_id`" but_meta="cm-dialog-opener" but_rel="nofollow"}

If you only wish to implement the fix to stop duplicate content for products.picker, categories.picker or both, you still need to add the 'but_rel' variable to the appropriate buttons.
Open:
/skins/your_skin/customer/buttons/button.tpl
Find:
<span class="button{$suffix}" {if $but_id}id="{$but_id}"{/if}><a {if $but_href}href="{$but_href|fn_url}"{/if}{if $but_onclick} onclick="{$but_onclick} return false;"{/if} {if $but_target}target="{$but_target}"{/if} class="{if $but_meta}{$but_meta} {/if}" {if $but_rev}rev="{$but_rev}"{/if}>{$but_text}</a></span>
Replace with:
<span class="button{$suffix}" {if $but_id}id="{$but_id}"{/if}><a {if $but_href}href="{$but_href|fn_url}"{/if}{if $but_onclick} onclick="{$but_onclick} return false;"{/if} {if $but_target}target="{$but_target}"{/if} class="{if $but_meta}{$but_meta} {/if}" {if $but_rel} rel="{$but_rel}"{/if} {if $but_rev}rev="{$but_rev}"{/if}>{$but_text}</a></span>

Refresh the site cache. Now all products.picker and categories.picker buttons/links have the 'nofollow' attribute. Wait for Google to re-crawl and re-love your site again.
Hire StellarBytes for CS-Cart Design & Development Services

 
  • johnbol1
  • Never Re
  • Members
  • Join Date: 23-Feb 10
  • 4671 posts

Posted 07 February 2013 - 05:24 PM #2

Cheers StellarBytes

John

Custom printed hi visibility clothing sale the UK's online hivis safety shop
v4.5.2


 

Posted 12 July 2013 - 05:52 PM #3

For the categories.picker Google turned up no results. For the products.picker it turned up about 994.

I'm on 3.0.6.

I guess I need to apply this fix....

Note that I've been using the canonical addon for years.

 
  • StellarBytes
  • Senior Member
  • Members
  • Join Date: 08-Aug 11
  • 1807 posts

Posted 12 July 2013 - 07:50 PM #4

Yes I sent the info to CS-Cart, the fix was supposed to be applied post-3.0.4 but was never applied in 3.0.5 or 3.0.6 but has been applied in V4. If you are on CS-Cart V2 or V3, you need to apply this fix.
Hire StellarBytes for CS-Cart Design & Development Services

 

Posted 12 July 2013 - 07:52 PM #5

Yes I sent the info to CS-Cart, the fix was supposed to be applied post-3.0.4 but was never applied in 3.0.5 or 3.0.6 but has been applied in V4. If you are on CS-Cart V2 or V3, you need to apply this fix.


You're a lifesaver. Thanks so much for providing this!

 

Posted 16 July 2013 - 04:50 PM #6

Hmmm. I went to make this change in the products_picker.tpl file, and the nofollow is already there.

So, looks like Google is not honoring the nofollow in this case?

Can this be fixed by adding a statement to my robots.txt file or not?

As I said above, I'm using 3.0.6 and have been since about the end of February.

 
  • The Tool
  • Been Here Way Too Long Member
  • Members
  • Join Date: 30-Mar 07
  • 3848 posts

Posted 16 July 2013 - 07:54 PM #7

Can this be fixed by adding a statement to my robots.txt file or not?


Yep. If I have said it once, I have said it a million times. If you have SEO add on enabled add:

Disallow: /index.php


 

Posted 20 July 2013 - 03:39 PM #8

Not sure what's going on with this, but Google now lists 1580 URLs for the products.picker.

 
  • boxed-blue
  • Junior Member
  • Members
  • Join Date: 15-Apr 11
  • 57 posts

Posted 08 August 2013 - 05:04 PM #9

Hi

I've tested both urls and get no results for both. The site is on v3.0.5

I've already added Disallow: /*? to the robots.txt file, this has resulted in a huge drop in indexed urls (the site has 2700k pages and had +85k indexed urls, most relating to url parameters being indexed (currencies etc). This hasn't stopped the strange urls I am getting:

/index.php?target=gift_certificates&mode=free_products&search_products=Y&page=322

Google WMT reports 404s that are linked to from urls of this nature. Google cannot even access these.

I'm trying the suggestion by The Tool... I'll let you know

Steve
CS Cart Design, Development and Hosting

This Is Kode - http://thisiskode.com

 
  • The Tool
  • Been Here Way Too Long Member
  • Members
  • Join Date: 30-Mar 07
  • 3848 posts

Posted 08 August 2013 - 10:05 PM #10

/index.php?target= hasn't been used since I think 1.3.x so the page should be returning a 404.

 
  • StellarBytes
  • Senior Member
  • Members
  • Join Date: 08-Aug 11
  • 1807 posts

Posted 08 August 2013 - 11:02 PM #11

Steve, exactly which version of the cart are you running? I assume by the fact you have "target=" URL's that this is the 1.3.5 store as per your signature. I have never touched V1 in any shape or form, so can't advise on that one.

Setting rel="nofollow" on links won't stop them being indexed if they are already indexed. Best to block them using .htaccess.

I actually looked at various sites and I am yet to see anyone actually buying a product for someone via gift certificates (ie. the product as a gift which results in products.picker URL's), so I removed the option to buy a product as a gift. Sorted. On the new stores I have launched where I have put this fix in place before going live, I haven't seen any products.picker nor categories.picker URL's being indexed in search engines.
Hire StellarBytes for CS-Cart Design & Development Services

 
  • boxed-blue
  • Junior Member
  • Members
  • Join Date: 15-Apr 11
  • 57 posts

Posted 09 August 2013 - 08:09 AM #12

Hi Stellar

The this particular site used to be 1.3.5 and moved up to v2.2.4 before going to 3.0.4 and finally 3.0.5

I've previously contacted CS Cart regarding the issue and they seemed to imply the links were still indexed. I blocked them in robots.txt a while back from accessing any urls with a ? in them and now blocked index.php from being indexed as a suggestion above.

The "target=" urls don't appear to be indexed and aren't accessible via browser yet are still showing up in Google WMTs as data under the "linked from" tab as though Google has accessed the page and found a 404 on that page.

I'm going to implement the .picker fixes today.

Cheers

Steve
CS Cart Design, Development and Hosting

This Is Kode - http://thisiskode.com

 
  • boxed-blue
  • Junior Member
  • Members
  • Join Date: 15-Apr 11
  • 57 posts

Posted 21 August 2013 - 04:12 PM #13

Hi All

Unfortunately we're still getting the strange urls albeit less than before.

A very strange issue indeed.

Steve
CS Cart Design, Development and Hosting

This Is Kode - http://thisiskode.com

 
  • StellarBytes
  • Senior Member
  • Members
  • Join Date: 08-Aug 11
  • 1807 posts

Posted 21 August 2013 - 05:58 PM #14

Can you give further examples of the URL structures, not the full URL as this will only encourage Google to crawl and index these URL's. Does Google Webmaster Tools show these strange URL's - if so, it should also show which page(s) the URL's are linked from.
Hire StellarBytes for CS-Cart Design & Development Services

 
  • boxed-blue
  • Junior Member
  • Members
  • Join Date: 15-Apr 11
  • 57 posts

Posted 23 August 2013 - 07:59 AM #15

Hi Stellar

This is the 404 from google webmaster tools.
/protective-motorcycle-clothing/motorcycle-clothing-by-type/motorcycle-helmets/visors/weise-waterproof-over-gloves.html

This is the link listed under the "Linked From" tab.
/protective-motorcycle-clothing/motorcycle-clothing-by-type/motorcycle-helmets/visors/index.php?target=gift_certificates&mode=free_products&search_products=Y&page=367

The 404 link is not even correct. The url structure for products is http://www.domain.co...oduct-name.html and this product is live and working. The category structure is correct but not for this product, this is a motorcycle helmet visor category and the product is a pair of gloves.

Google is blocked in the robots.txt file from crawling any urls with a ? in them.

Thanks for coming back. Hope this helps.

Cheers

Steve
CS Cart Design, Development and Hosting

This Is Kode - http://thisiskode.com

 
  • vanderex
  • Senior Member
  • Members
  • Join Date: 27-May 10
  • 386 posts

Posted 20 February 2014 - 07:24 PM #16

this not working as im still getting spidered with pickers.....

4.03
Get Cost Effective Content Delivery CDN
Unbeatable speed, stats, & price
MaxCDN Site Acceleration

 
  • meetshamee
  • Advanced Member
  • Trial users
  • Join Date: 12-Feb 13
  • 92 posts

Posted 04 March 2014 - 02:09 PM #17

this not working as im still getting spidered with pickers.....

4.03


any solutions?

 
  • boxed-blue
  • Junior Member
  • Members
  • Join Date: 15-Apr 11
  • 57 posts

Posted 12 March 2015 - 08:42 AM #18

I've recently upgrade the store mentioned above to v.4.2.4 and Google WMTs is still reporting 404s for urls including /index.php?target=gift_certificates

I've contacted CS Cart previously only to be told target isn't being used in CS Cart anymore, which I know that.

Is anyone else getting this?

Steve
CS Cart Design, Development and Hosting

This Is Kode - http://thisiskode.com

 
  • CartBooster
  • Senior Member
  • Members
  • Join Date: 25-Jun 06
  • 246 posts

Posted 09 February 2016 - 12:23 PM #19

Hi,

 

Just an update, we're on 4.3.5 and one of our website has this issue,

Any suggestions on how to fix it for this branch?

 

Stephen


www.cartbooster.ie


 
  • StellarBytes
  • Senior Member
  • Members
  • Join Date: 08-Aug 11
  • 1807 posts

Posted 13 February 2016 - 11:34 PM #20

Hi,

 

Just an update, we're on 4.3.5 and one of our website has this issue,

Any suggestions on how to fix it for this branch?

 

Stephen

It's been a while since I experienced this problem, but can happily take a look for you if you wish.


Hire StellarBytes for CS-Cart Design & Development Services