Jump to content

  • You cannot start a new topic
  • You cannot reply to this topic

Var/cache/registry - 10's Of Thousands Of Folders - Host Is Not Happy. Rate Topic   - - - - -

 
  • grafis
  • Junior Member
  • Members
  • Join Date: 12-Jan 11
  • 150 posts

Posted 10 April 2015 - 01:29 PM #1

CS-Cart 4.1.5 site with 100 categories and 880 products is generating files & folders in /var/cache/registry all day long day in day out. I found over 20,000 folders when I checked. It appears either that process or search engine bots crawling it got us flagged for high CPU/Memory use by our host.

Any thoughts on how to to stop this? I've tried the steps below but still have a high CPU/Memory issue and thousands of folders are still being created. CSC has created 1,935 /cache/registry folders in the last 8hrs and is still going strong - blocks_content_xx...., thooks_xx..., schema_xx.

Filters - I found that the Price and Brand sidebar blocks in the Layout section had automatically created "filters" which in turn generated thousands upon thousands of pfilter folders - so I disabled the Filters.
Layout Blocks - I then found the layout blocks seem to generate cache for each and every anything - so I've paired those down to minimum.
Robots.txt - I had Dissallow: /var/ as but changed to Disallow: /*features_hash per some posts in these forums (no idea what that is).
Webmaster tools - I told both Google and Bong to calm down and crawl at a slow rate (and already had a sitemap defined for both).
Currency - three currencies active.
Database - optimized.
Server - checked for errors on the server etc etc.

Desperate as it's a small site with limited traffic (100 visitors a day) and we're about to get kicked to expensive next level hosting. I have two other CSC sites with far more traffic and 1/10th of the resource consumption.

CS-Cart 4.7.1


 
  • chrisbti
  • Senior Member
  • Members
  • Join Date: 20-Oct 11
  • 262 posts

Posted 10 April 2015 - 04:46 PM #2

Obviously you can delete the cache folder daily located ' /var/cache ' or install a cron app to do it for you. Maybe CS v. 4.3 will also do this.

I have about the same amount of products on my site . this is what I've been doing. Most e-commerce carts do the similar cache storage.

You may also add
Disallow: /*? to robots.txt

Regards, God Be With You! Jesus Christ our Lord & Savior.
BTI - Better Tools For Industry
Industrial Hand Tools Made in USA | www.BTI-Tool.com | v.4.9.3.SP1 | Apache


 

Posted 10 April 2015 - 05:06 PM #3

if you really worry about number of folders and server where cs-cart installed is not a shared one, you may try to use redis cache (file cache is enabled by default).
the redis type of cache requires installation of Redis on your server and it works faster than file cache so instead of generating cache files, data are saved in server memory (in Redis)
here is article about how to change type of cache

best regards,
WSA team

 
  • grafis
  • Junior Member
  • Members
  • Join Date: 12-Jan 11
  • 150 posts

Posted 10 April 2015 - 06:24 PM #4

Thanks @chrisbti - Good to know the 10,000's of folders isn't uncommon (but drastically different than my two other sites that have ~800). Deleting daily - it's been over 8 hours since I manually deleted all folders last. CSC has been rebuilding the folders ever since, currently at 2,430 folders and still increasing at a steady rate of ~10 every 3-5 minutes. Not sure if it will be done within 24hrs or not and if I delete it starts all over again and I assume will sock CPU/RAM.
I'll try adding /*? to Robots.

Thanks @WSA - I'm worried about CPU and RAM consumption which appears to be caused by the creation of the files and folders or by SE Bots crawling the structure. I'll check out Redis to see if it will help.

CS-Cart 4.7.1


 
  • grafis
  • Junior Member
  • Members
  • Join Date: 12-Jan 11
  • 150 posts

Posted 12 April 2015 - 07:07 PM #5

Solved it by disabling Block cache in config.local.
// Tweaks
$config['tweaks'] = array (
    'disable_block_cache' => true, // used to disable block cache

Look like Redis would be the way to go, with a good bit of work.

CS-Cart 4.7.1


 
  • eComLabs
  • CS-Cart Expert
  • Authorized Reseller
  • Join Date: 27-Jan 14
  • 19158 posts

Posted 13 April 2015 - 11:06 AM #6

Solved it by disabling Block cache in config.local.

// Tweaks
$config['tweaks'] = array (
	'disable_block_cache' => true, // used to disable block cache

Look like Redis would be the way to go, with a good bit of work.


Note that the website will work slowly with disabled blocks cache. Did you contact CS-Cart team with this issue?

GET A FREE QUOTE | CS-Cart Add-ons | CS-Cart Licenses | CS-Cart Development | CS-Cart Design | Server Configuration | UniTheme and YOUPI
CS-Cart                USD 345     Multi-Vendor              USD 1250    CS-Cart RU                         24500 руб.
CS-Cart Ultimate  USD 775     CS-Cart + YOUPI      USD 545      CS-Cart RU + UniTheme    36000 руб.


 
  • grafis
  • Junior Member
  • Members
  • Join Date: 12-Jan 11
  • 150 posts

Posted 13 April 2015 - 12:32 PM #7

Note that the website will work slowly with disabling blocks cache. Did you contact CS-Cart team with this issue?


Yep but the immediate goal was to reduce resource usage and it did instantly. The site only gets about 100 visitors a day and appears to be just as fast to load as ever (I notice no difference) and still gets a 95 page speed and 91 Yslow score on GTmetrix. I realize that if the site were under heavy load lack of caching blocks would impact performance.

I will be in touch with support - I migrated the site from Miva Merchant so my guess is there is something wrong in the DB, the question is why suddenly out of the blue.

CS-Cart 4.7.1


 
  • eComLabs
  • CS-Cart Expert
  • Authorized Reseller
  • Join Date: 27-Jan 14
  • 19158 posts

Posted 13 April 2015 - 03:46 PM #8

For several customers we added a script to the cron task which clears the cache at the time when there is a minimum amount of visitors (e.g. at 3:00 am)

GET A FREE QUOTE | CS-Cart Add-ons | CS-Cart Licenses | CS-Cart Development | CS-Cart Design | Server Configuration | UniTheme and YOUPI
CS-Cart                USD 345     Multi-Vendor              USD 1250    CS-Cart RU                         24500 руб.
CS-Cart Ultimate  USD 775     CS-Cart + YOUPI      USD 545      CS-Cart RU + UniTheme    36000 руб.


 
  • grafis
  • Junior Member
  • Members
  • Join Date: 12-Jan 11
  • 150 posts

Posted 13 April 2015 - 09:11 PM #9

For several customers we added a script to the cron task which clears the cache at the time when there is a minimum amount of visitors (e.g. at 3:00 am)


Unless I'm missing something I believe that would increase our resource consumption not reduce it. Our excessive resource usage issue appears directly tied to the creation of the cache folders. Anytime I manually deleted the /var/registry cache we then saw a spike in resource usage as CS-Cart created folders and files at the rate of about 5-10 folders every five minutes and had only created 2,400 folders out of xxx??? after 24hrs - in this case prospectively using 10-20x's our allocated resources for 2-5 days in order to create all of the folders.

CS-Cart 4.7.1


 
  • WebTek
  • Member
  • Authorized Reseller
  • Join Date: 10-Oct 07
  • 109 posts

Posted 22 May 2015 - 01:42 AM #10

I have the same problems with literally millions of files in the var/cache area. I also noticed filters adding tons of entries in the cache. Is there a way to disable filter caching or ALL caching? I temporarily disabled filters, but really want to use them.

 
  • eComLabs
  • CS-Cart Expert
  • Authorized Reseller
  • Join Date: 27-Jan 14
  • 19158 posts

Posted 22 May 2015 - 09:45 AM #11

I have the same problems with literally millions of files in the var/cache area. I also noticed filters adding tons of entries in the cache. Is there a way to disable filter caching or ALL caching? I temporarily disabled filters, but really want to use them.


Please disable filter URLs in the robots.txt file. Make a search on the forum

GET A FREE QUOTE | CS-Cart Add-ons | CS-Cart Licenses | CS-Cart Development | CS-Cart Design | Server Configuration | UniTheme and YOUPI
CS-Cart                USD 345     Multi-Vendor              USD 1250    CS-Cart RU                         24500 руб.
CS-Cart Ultimate  USD 775     CS-Cart + YOUPI      USD 545      CS-Cart RU + UniTheme    36000 руб.


 
  • WebTek
  • Member
  • Authorized Reseller
  • Join Date: 10-Oct 07
  • 109 posts

Posted 22 May 2015 - 12:40 PM #12

The only thing I found in the forums is to add "Disallow: /*features_hash" to the robots.txt file, and not sure what that does. I added it, enabled the filters and the cache starting running wild with hundreds of new filter entries every minute.

Is there a way to use filters (on the front-end), but completely disable filter caching? ... and/or completely not use caching?

 
  • cmund
  • Member
  • Members
  • Join Date: 01-Jun 10
  • 129 posts

Posted 19 January 2017 - 01:04 PM #13

Can EVERYTHING in the var/cache/registry be deleted?  I have a large file called:  cache_update_handlers 

 

I want to make sure before I delete it.  I'm having the same issues with large amounts of file generated.  I'm waiting to hear back from the help desk but in the meantime, I want to alleviate my issues as my host is not happy either.


Version 4.2.4


 
  • Darius
  • Douchebag
  • Members
  • Join Date: 20-Apr 08
  • 3246 posts

Posted 19 January 2017 - 01:23 PM #14

clear cache from admin "admin.php?cc&ctpl"

basically means deleting everything in

 

var/cache/

 

second you delete all in

var/cache/

 

new

var/cache/registry

and others will be created..



 
  • cmund
  • Member
  • Members
  • Join Date: 01-Jun 10
  • 129 posts

Posted 19 January 2017 - 02:01 PM #15

Thanks Darius


Version 4.2.4


 
  • eComLabs
  • CS-Cart Expert
  • Authorized Reseller
  • Join Date: 27-Jan 14
  • 19158 posts

Posted 19 January 2017 - 02:03 PM #16

Can EVERYTHING in the var/cache/registry be deleted?  I have a large file called:  cache_update_handlers 

 

I want to make sure before I delete it.  I'm having the same issues with large amounts of file generated.  I'm waiting to hear back from the help desk but in the meantime, I want to alleviate my issues as my host is not happy either.

 

Yes, the var/cache directory can be deleted at any time. Note that the store will be slow first time


GET A FREE QUOTE | CS-Cart Add-ons | CS-Cart Licenses | CS-Cart Development | CS-Cart Design | Server Configuration | UniTheme and YOUPI
CS-Cart                USD 345     Multi-Vendor              USD 1250    CS-Cart RU                         24500 руб.
CS-Cart Ultimate  USD 775     CS-Cart + YOUPI      USD 545      CS-Cart RU + UniTheme    36000 руб.


 
  • cmund
  • Member
  • Members
  • Join Date: 01-Jun 10
  • 129 posts

Posted 19 January 2017 - 04:22 PM #17

Thank you eComLabs.


Version 4.2.4