.htaccess robots.txt

Business Marketing & SEO
1

Hi Everyone,



How we have to form our .htaccess and robots.txt files? I think they're not confidential and almost all of the rules in the files can be widely used for everyone.So:


  • Can you post your htaccess file for your store directory & main directory and robots.txt for the reference.



    We will create these required files before getting our site indexed by Google. We have to be sure that we don't miss anything to be included.





    Thanks,
2

Anyone there to help?

3

I doubt that anyone is going to post their .htaccess file blindly.

The standard file works perfectly for me and I've added some of the gzip stuff to compress the output.

Standard robots.txt is okay too with exception of I've added Yahoo and GoggleBot throttles to only allow 2 pages/minute to be read by a crawl. Takes longer to get an initial index, but sure reduces the load on the server.