Pre Spider Website

I notice the initial load is always slow after clearing cache (i.e. ?cc) on all pages including home page, category page, and product page. Once you load one category, one home, and one product page the speed gets better. I can't tell for sure if loading additional pages for example multiple categories, and product pages would speed up loading time for the next end user.

Would there be any benefit to creating a spider that pre-spiders the website essentially loading all the category, product, etc… pages into the cache or does the CS-Cart cache not work this way?

Spend your time on product marketing. Much better return on your investment of time.

But performance is an issue with all the visitors, so wouldn't you want it working optimized for the customer before marketing a ton?

If you're clearing your cache that frequently, then something else is wrong. Cache is cleared automatically by certain operations such as editing an addon, etc.

You can do what you propose, I'm just suggesting the return on investment is low. But some people like to fiddle…

No not that frequently. It might happen once a week automatically when we process a product database update using the addon we have build but I would hate a user to have to wait for a full compile which can take 3-5 seconds. As you know 3-5 seconds doesn't sound long but to the customer its FOREVER.