Jump to content

  • You cannot start a new topic
  • You cannot reply to this topic

Meta Description Blocked By robots.txt Rate Topic   - - - - -

 
  • boxed-blue
  • Junior Member
  • Members
  • Join Date: 15-Apr 11
  • 59 posts

Posted 23 August 2013 - 02:39 PM #1

Dear All

I'm seeing an issue with a clients website where the description in search engine results is being blocked by the robots file.

I have used webmaster tools to test this and they are not being blocked by the robots.txt file. The only notable aspect we have altered is adding an CS Cart seo addon to add a canonical url to a possible duplicate product.

robots.txt
User-agent: *
Disallow: /*?
Disallow: /index.php
Disallow: /images/thumbnails/
Disallow: /skins/
Disallow: /payments/
Disallow: /store_closed.html
Disallow: /core/
Disallow: /lib/
Disallow: /install/
Disallow: /js/
Disallow: /schemas/
Thanks in advanced

Steve

CS Cart Design, Development and Hosting

This Is Kode - https://thisiskode.com


 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 12156 posts

Posted 23 August 2013 - 06:08 PM #2

robots.txt controlls access to 'files'. It does not control the parsing of those files.

EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • boxed-blue
  • Junior Member
  • Members
  • Join Date: 15-Apr 11
  • 59 posts

Posted 24 August 2013 - 10:31 AM #3

Yeah its a very odd one although Googling the issue throws up a fair amount of discussion on it especially related to Wordpress. I feel its nothing to do with robots what so ever.

CS Cart Design, Development and Hosting

This Is Kode - https://thisiskode.com