Googlebot sent me an email and it said that it does not like cs-cart's url structure

General General Questions
1

Hi



I just got emailed from Googlebot #24256 version 1000.02 sector 2 and it said the following about cs-carts url structure:



Googlebot encountered problems while crawling your site [url]http://www.XXXXXXXXXXXXXXX.com/[/url].



Googlebot encountered extremely large numbers of links on your site. This may indicate a problem with your site’s URL structure. Googlebot may unnecessarily be crawling a large number of distinct URLs that point to identical or similar content, or crawling parts of your site that are not intended to be crawled by Googlebot. As a result Googlebot may consume much more bandwidth than necessary, or may be unable to completely index all of the content on your site.



Examples:



[url]http://www.mydomain.com/index.php?dispatch=auth.login_form&return_url=index.php%3Fdispatch%3Dcategories.view%26category_id%3D93%26subcats%3DY%26features_hash%3DP12.V78.V21[/url]

[url]http://www.mydomain.com/wda/?subcats=Y&features_hash=V276.V67.V721[/url]

[url]http://www.mydomain.com/asd/casdda/?subcats=Y&features_hash=V331.V349.P15[/url]

[url]http://www.mydomain.com/wasd/?subcats=Y&features_hash=V767.V246.V276.V617[/url]

[url]http://www.mydomain.com/asd/?subcats=Y&features_hash=V261.V230.V274.P18[/url]

[url]http://www.mydomain.com/asdsadas/?subcats=Y&features_hash=V20.V119.P18[/url]

[url]http://www.mydomain.com/asdsad/asdas/page-6/?subcats=Y&features_hash=V734.V216.V552[/url]

[url]http://www.mydomain.com/index.php?dispatch=auth.login_form&return_url=index.php%3Fdispatch%3Dcategories.view%26category_id%3D173%26subcats%3DY%26features_hash%[/url]



Thank you



Googlebot #24256 version 1000.02





---------------------





How to solve this?



Thanks



B

2

Hey cs-cart, google bot (Matt Cutts revision) sent me a link on how to fix this problem…



Please fix



[url]http://www.google.com/support/webmasters/bin/answer.py?answer=76401[/url]

3

All search form links and login links should have the “nofollow” attribute set.

4

Looks like it is trying to index your product filters as well from the category pages. I changed the code in the version of cs cart that I am running to have nofollow attributes on all of the category product filters.

5

All filters, searches, features and login links should have nofollow set. Otherwise the robots will just chew up your site and never get anywhere.

6

[quote name=‘tbirnseth’]All filters, searches, features and login links should have nofollow set. Otherwise the robots will just chew up your site and never get anywhere.[/QUOTE]



I’m curious is this something you have to input in the cart or does it come standard with the nofollow command?



Thanks

7

It “should” come standard with nofollow but it doesn’t.



Hence you need to track down all the anchor tags in the templates that are links to the types of things as listed above and add the following to the anchors that don’t reference useful info. Add:

rel=“nofollow”



to the anchor tags for login, searches, filters, product filters and probably all profile access and other stuff that simply has more links or takes someone to a form (Ie. a not a content page).

8

Hey cs-cart, please fix in next version…



Thanks

9

If you want it fixed you should post it as a bug in the bugtracker. They don’t read this stuff…