Hi,
When I check out my stats sometimes I see Google crawling my pages, 2 per day, 5 per day, sometimes 20 or so, I even had once where it crawled 200 plus.
Does this sound about right or should I be seeing hundreds per time or any ideas about what I could expect. I have uploaded my content and site map, confirmed my site etc etc.
Thanks
John
It varies. 200 is low depending on how many pages you have. What stats are you checking? Use Google Webmaster stats to give you a better idea.
Also check how many seconds per page you have it configured to crawl. Slower rates will cause it to give up early and faster rates may impact your site’s performance. You can also specify the rate you want in the robots.txt file with the directive:
Request-rate: 10
Which says to do one page every 10 seconds. This would equate to a “crawl rate” in your Google webmaster account of “.1”.
thx for info.
John
I have a few different sites and it seems to vary greatly on how many items your store has.
For example:
Store 1: 750 products and Google crawls 206 average per day
Store 2: 109,000 products and average crawl rate is 3011 per day
Not sure if this helps give you and idea, but it might.
Admin stats in CS show sometime only 2 sometime 200 or so, but site has 3000 plus products,
Google says high crawl is 1800 fo rmy site and low is 200 (actually crawled) so basically I shall ignore the CS stats then?
JOhn