[quote]
We would like to show you a description here but the site won’t allow us.
[/quote]
This is showing as the description for my site in Yahoo search results. Can anyone tell me why?
robots txt?
Is meta info also in homepage
This is what's in my robots.txt - not sure what I'm looking for that would cause this:
User-agent: *
Disallow: /images/thumbnails/
Disallow: /app/
Disallow: /design/
Disallow: /js/
Disallow: /var/
Disallow: /store_closed.html
I do have my meta info entered as well.
See here and here, it may help. Not 100% sure if CS-Cart fixed these issues in V4.0.2 or not, but they were reported by quite a few people on the forum with 4.0.1.
CS-Cart has implemented the change in the 2nd link already.
Also I just saw this in Google's results, so it is the robots.txt but I don't know what to change in that file:
The fix was to remove the “Disallow: /var/” line, however, this may potentially open you up to potentially harmful indexing of /var/ directories (in particular the database backups, but I believe this issue is partially resolved via .htaccess rules within those directories). I would love to say contact CS-Cart for advice on this, but in my experience CS and SEO go together like lamb and tuna fish.
I would assume the only 'cache' directories mentioned in that thread which are problematic for search engines is /var/cache/ - therefore I would be more hesitant to allow access to the entire /var/ directory and only allow access to the /var/cache/ directories and still disallow to the other folders (/var/database/, /var/exim/, etc.)
In essence, your robots.txt would need to be appended similar to:-
Allow: /var/cache/[quote name='StellarBytes' timestamp='1383002939' post='170518']
The fix was to remove the “Disallow: /var/” line, however, this may potentially open you up to potentially harmful indexing of /var/ directories (in particular the database backups, but I believe this issue is partially resolved via .htaccess rules within those directories). I would love to say contact CS-Cart for advice on this, but in my experience CS and SEO go together like lamb and tuna fish.
I would assume the only 'cache' directories mentioned in that thread which are problematic for search engines is /var/cache/ - therefore I would be more hesitant to allow access to the entire /var/ directory and only allow access to the /var/cache/ directories and still disallow to the other folders (/var/database/, /var/exim/, etc.)
In essence, your robots.txt would need to be appended similar to:-
Allow: /var/cache/
[/quote]
HA! Thanks for the tuna fish link :OP
I will try this & see what happens. Thanks!
Stellar, why would a search engine need access to anything the the var directory? The cache is served by PHP and is never seen by the front end. All your file uploads (and client uploads) would then be visible. Please explain exactly what you believe the robot is looking at/for in var. Now if there's a link on a page that directly reference var/* then yes, could be a problem. But the problem (I think) would be creating static content in the var directory tree.
So should I put that back in there?
Brennie, not sure what you mean by put it back… The controller does a number of things with this data. First it verifies that it has been created within the timeframe that it's setup to do and if not, recreates it. If you have moved it to the root of your store then you should remove it or it will become 'stale' and you'lll never get to the correct xml file. So I guess the short answer is that I'd just delete it and let it be recreated on next access.
I was referring to the robots.txt file. Google (& Yahoo and probably more) are showing my pages are non-accessible, and my sitemap as well.
Do not know what you've done. I would use the standard supplies robots.txt file as a starting place. Verify it works for most things and then make adjustments as you think you need to.
Tony, your understanding of /var/ is the same as my own, I couldn't understand why this seemed to be the case, but I've seen a number of reports on the forum and the bugtracker where the solution was either the fix in the Statistics addon or by allowing access to /var/ which I don't think is a good idea, I can only assume there must be something within /var/ or subdirectories of /var/ which search engine spiders need access to in some shape or form. Only the /var/cache/ folder seemed to make even the slightest bit of sense, but that in itself doesn't make sense!