I received the following warning from my Google Product listings today, and was hoping someone has had similar issue or could give me some detailed assistance. Here is the message:
[COLOR=“DarkRed”]Thank you for participating in Google Product Search. It has come to our attention that a robots.txt file is preventing us from crawling some or all of the images on your site. In order for us to access and display the images you provide in your product listings, we’d like you to modify your robots.txt file to allow user-agent ‘googlebot’ to crawl your site.
Failure for Google to access your images may affect the visibility of your items on Google Product Search and Product Ad results.
To ensure the ‘googlebot’ is not being blocked, please add the following two lines of text to the end of your robots.txt file:
User-agent: googlebot
Disallow:
[/COLOR]
Any ideas? Thanks!
Joe,
By default the robots.txt file blocks Google from searching your thumbnails. I just checked your file and that is the case with your robots.txt.
Google can still index your detailed images, just not your thumbnails.
To be honest, I don’t know if it matter either way. It’s easy enough to change, but it would be interesting to know if it matters. Hopefully someone else will add to this.
Thanks,
Brandon
It depends on which path is being use for the image you are specifying to Google. If it is in the images/thumbnails tree then they will honor the Robots.txt file saying Disallow.
Personally, referencing a file from a specified image path is quite different than indexing a directory of images. They should not be requiring you to lift crawl permissions so they can access a fully qualified URL. But then they’re Google and they can force you to do whatever they want you to do.
Thank you both for your responses. That being said, should I make any modifications to the file, or just let it ride and see what happens?
Google is my main source of traffic & sales, so I would hate to lose the feed for any period of time.
If you don’t have an issue with your images being indexed (and then available on Google Images) then just leave it (my opinion, others may differ). But if you don’t want your images indexed outside your products, make sure you have thumbnails for everything and then have it use the thumbnail image.
I posted a thread sometime ago asking what the reason for disallowing thumbnails was. Guess how many responses it got? 0
If you want Googlebase or Bingshopping to display your products then you need to edit/remove that line in robots.txt.
You’re just ahead of your time! 
[quote name=‘The Tool’]I posted a thread sometime ago asking what the reason for disallowing thumbnails was. Guess how many responses it got? 0
If you want Googlebase or Bingshopping to display your products then you need to edit/remove that line in robots.txt.[/QUOTE]
Did you delete the entire string Disallow: /images/thumbnails/ or just alter the values? Thanks for your help.
[quote name=‘Chef_Joe’]Did you delete the entire string Disallow: /images/thumbnails/ or just alter the values? Thanks for your help.[/QUOTE]
I deleted that line.
it’s oki for you now ? your product accepted ?
I removed the 1 line of text as indicated above and I have not received any more warning emails from Google.
I’d like to assume that the issue has been solved. I also just checked my feed and all images are present and attributed to the correct product.
Hopefully this issue is solved. Thanks everyone for your input!