2 questions in 1 really,
google is spending a lot of times crawling pages like this
http://www.hivis.co.uk/index.php?dispatch=products.picker&display=options&holder_name=gift_certificates&page=184
Dont see the point and am wondering if it may count them as duplicate content if they are just the search pages.
Can I / should I stop them being crawled anyway.
Thanks
John
Would find what link they are being initiated from and then simply ensure there is a 'rel=“nofollow”' attribute to the anchor tag. This “should” keep the crawlers from following the link.
In about 2.1, they added nofollow to many of the links like login page, feature comparison, etc. that really do nothing to help a search engine. But it's possible they missed some.
Why not just block “index.php?dispatch=” in robots.txt?
Because then you might not get your products listed for those that may not have SEO names set.
[quote name='tbirnseth' timestamp='1324153321' post='127986']
Because then you might not get your products listed for those that may not have SEO names set.
[/quote]
It doesn't matter. If the SEO addon is turned on and the product is active, there will be some kind of name regardless. Even if there is no product name, the SEO name will be “product-(id #)”.
This problem still exists and seems a widespread problem and I can't seem to find the source of the 'products.picker' being indexed in 2.2.4.
Anyone?
Update: Found a suitable fix and have posted the products picker and categories picker duplicate content fix here