Basically. robots.txt points from the wrong directory for me… “/usr/local/apache/htdocs/robots.txt” which is not available on my current server. I did generate a “robots.txt” file under the document root of “/home/myservername/public_html” and need to configure it to work. How can I tell CS-Cart to look at /home/myservername/public_html instead of /usr/local/apache/htdocs/ for the file?
CS-Cart: version 3.0.6 PROFESSIONAL
Thanks.
Don't believe Cs-Cart even looks at the robot.txt file… Could be wrong.
Mostly for search engine bots…
just drop the robots.txt where you need it to be in root, and add what info in it you need.
JOhn
I don't get it. The hesk desk referenced a default file which I wrote about here.
Maybe the default file is somehow listed at [font=“arial, verdana, tahoma, sans-serif”][color=“#282828”]/usr/local/apache/htdocs/robots.txt which is not available on my current hosting setup. I use a VPS, btw. This is really the first I've ever heard of this being an issue. My host just got back with about my server error log which shows multiple field attempts to access [/color][/font][color=#282828][font=arial, verdana, tahoma, sans-serif]/usr/local/apache/htdocs/robots.txt which the hosting company says, DOES NOT EXIST and that I need to get this updated with CS-Cart. Really baffled.[/font][/color]
check your robots.txt by going to Fastest Web Hosting Services | Buy High Quality Hosting
that is where it is supposed to be. What is looking in htdocs (your local path btw) do you know what is trying to access it, it will show a reference in cpanel error logs.
John
Thanks for getting back with me johnbol1
This is what I'm seeing on my end:
[Wed Dec 25 15:37:32 2013] [error] [client 66.249.76.209] File does not exist: /usr/local/apache/htdocs/robots.txt
[Wed Dec 25 23:12:42 2013] [error] [client 141.101.80.254] File does not exist: /usr/local/apache/htdocs/robots.txt
[Fri Dec 27 07:05:30 2013] [error] [client 173.245.56.28] File does not exist: /usr/local/apache/htdocs/robots.txt
[Fri Jan 03 07:12:28 2014] [error] [client 141.101.80.254] File does not exist: /usr/local/apache/htdocs/robots.txt
According to Google, client 66.249.76.209 is crawl-66-249-76-209.googlebot.com. The other two ip addresses appear to be related to CloudFlare. It looks like both are trying to access the robots.txt file from /usr/local/apache/htdocs/.
[quote name='ckad79' timestamp='1389042945' post='174633']
Thanks for getting back with me johnbol1
This is what I'm seeing on my end:
[Wed Dec 25 15:37:32 2013] [error] [client 66.249.76.209] File does not exist: /usr/local/apache/htdocs/robots.txt
[Wed Dec 25 23:12:42 2013] [error] [client 141.101.80.254] File does not exist: /usr/local/apache/htdocs/robots.txt
[Fri Dec 27 07:05:30 2013] [error] [client 173.245.56.28] File does not exist: /usr/local/apache/htdocs/robots.txt
[Fri Jan 03 07:12:28 2014] [error] [client 141.101.80.254] File does not exist: /usr/local/apache/htdocs/robots.txt
According to Google, client 66.249.76.209 is crawl-66-249-76-209.googlebot.com. The other two ip addresses appear to be related to CloudFlare. It looks like both are trying to access the robots.txt file from /usr/local/apache/htdocs/.
[/quote]
/usr/local/apache/htdocs/ is indeed the root of your site and is indeed where your robots.txt SHOULD go. CS-Cart has very little to do with your robots file. The file basically tells “robots” where and where not to look on your site.
did you check the Fastest Web Hosting Services | Buy High Quality Hosting if it shows error 404 drop the robots file in there and that should sort it.
John
[quote name='johnbol1' timestamp='1389046669' post='174637']
did you check the Fastest Web Hosting Services | Buy High Quality Hosting if it shows error 404 drop the robots file in there and that should sort it.
John
[/quote]
When I go to [color=#282828][font=arial, verdana, tahoma, sans-serif]Fastest Web Hosting Services | Buy High Quality Hosting, it shows the robots.txt file with no issues. [/font][/color]