Sitemap And Robots...

...we were on 2.2.x for a LONG time until our host needed to upgrade PHP this past December. Therefore, we had CSCart support do the upgrade for us to the latest 4.7.1SP2 release. Everything worked out well, extremely happy with their help, and it saved me a ton of time.

I am still tweaking/learning everything, and I see that 1) I have lost my old robots.txt file (which probably wouldn't be applicable anyway), and 2) I am getting the usual "sitemap is html" error from Google. Doing as much looking as I can, I don't see clear answers here, and I am not fluent in Russian. So :

- what would be a good starting point for a robots.txt file?

- what can be done about the sitemap issue (have SEO and sitemap add-ons turned on)

Thanks for any help or redirects to answers.

...we were on 2.2.x for a LONG time until our host needed to upgrade PHP this past December. Therefore, we had CSCart support do the upgrade for us to the latest 4.7.1SP2 release. Everything worked out well, extremely happy with their help, and it saved me a ton of time.

I am still tweaking/learning everything, and I see that 1) I have lost my old robots.txt file (which probably wouldn't be applicable anyway), and 2) I am getting the usual "sitemap is html" error from Google. Doing as much looking as I can, I don't see clear answers here, and I am not fluent in Russian. So :

- what would be a good starting point for a robots.txt file?

- what can be done about the sitemap issue (have SEO and sitemap add-ons turned on)

Thanks for any help or redirects to answers.

http://docs.cs-cart.com/4.7.x/user_guide/addons/google_sitemap/set_google_sitemap.html

http://docs.cs-cart.com/4.7.x/user_guide/website/robots.html

Thanks, mart.

Robots - yes, I am aware of how to construct a robots file, but wondering what the recommendations are for what folders/files to disallow. My previous robots (wherever that went) had lines that likely pertained to outdated 2.2.x folders/file, and would not be applicable to 4.7.1 anyway. ?

Sitemap - yes, have sitemap going, and have SEO enabled as mentioned. But, Google webmaster still gives me the error. According to the forum post you linked, with SEO enabled, I shouldn't have to edit htaccess (nor do I want to).

Still wondering, and thanks again!

One last plea for the sitemap issue (haven't figured out the robots but I will).

Still getting a warning in Google's console about the sitemap appearing to be an html page. SEO add-on is enabled and working. Link/url to my sitemap.xml file loads properly on my end within a browser. Mostly stock CSCart installation which is why I assume someone else has had this issue, but see nothing in the forums (in english).

Weird bug in my (recent) upgrade/installation, or something simple I am missing? Thanks again for any help.

Please provide us with the fill text of error message from Google

Thanks, eCom, should have posted that originally. Stupid! Latest I see in console :

"Sitemap is HTML Your Sitemap appears to be an HTML page. Please use a supported sitemap format instead. Line 3 Feb 1, 2018"

I have a ticket in at CS, just thought I would also post here in case others have the same issue.

Thanks again.

What do you see if you enter the following URL in your browser?

http://your_domain.com/sitemap.xml

this happened to me and seemed to be an errors in WM tools, deleting and re submitting my sitemap a few times worked

Thank you eCom and john, much appreciated. I think I stumbled onto something I didn't notice before...

When I load https://www.domain.com/sitemap.xml, it loads our home page.

When I load https://domain.com/sitemap.xml, it loads our sitemap.

Hmm. I have always had https://www.domain.comlisted for the property in Google's tools, but should I have (instead or in addition to) a property for https://domain.com? I don't think that should matter.

Possibly related now that I look at site settings in Google, I see the preferred domain setting is set for no "www", and I have always had no preferred domain. I try to change it to no preferred domain, but it doesn't save. Not sure if this is related, or why I cannot change. ?

Odd, but it seems CSCart may be working fine, as I assumed, and it is my settings in Google causing an issue... ?

Ok, so I now have in my Google console the following 4 properties (whether I need them or not), resubmitted sitemap.xml in all of them, and what the sitemap status is :

http://ourdomain.com- sitemap is reporting properly

https://ourdomain.com- sitemap is reporting properly

http://www.ourdomain.com- sitemap reporting the html error

https://www.ourdomain.com- sitemap reporting the html error

(After these 4 were in place, I was also able to save the "no preferred domain" site settings in each. Again, no idea what that will do.)

So, at least I know my CS installation is not causing anything, at least I don't think. I will call them off. And thank you for pointing me in the right direction of loading different URL's for the sitemaps to test, must have missed that the first 50 times I tried.

Now, just not sure what properties above I really need to maintain in Google console. ?

I think you still have a redirect issue? The www should redirect to the non www site.

Tool - thanks for that, yes, still wondering that myself as in my prior post about loading the home page instead of the sitemap. Odd.

Make sure you have all the same domains in Store settings and in /config.local.php.

Small update but no real progress. I received a "stock" htaccess file from CS support, but it's quite a bit different than the one I currently have, and our store is working fine other than this sitemap issue, so not sure I want to mess with it.

Tool - thank you. config.local has "domain.com" (no www), so not sure what that gets me or if that's correct. Again, almost sure that is what I have always had.

Bottom line is that the url domain.com/sitemap.xml loads fine, but www.domain.com/sitemap.xml loads our home page. And maybe it's no big deal and won't make any difference with Google or SEO, I'm no expert there.

Thanks everyone for the feedback and help, if I come to any other conclusions as to what we're doing wrong, I'll reply back.

Bottom line is that the url domain.com/sitemap.xml loads fine, but www.domain.com/sitemap.xml loads our home page. And maybe it's no big deal and won't make any difference with Google or SEO, I'm no expert there.

Did you try to contact hosting administrator with this issue?