Jump to content

  • You cannot start a new topic
  • You cannot reply to this topic

Improved caching system Rate Topic   - - - - -

 
  • TexasGuy
  • Senior Member
  • Members
  • Join Date: 18-Mar 10
  • 546 posts

Posted 27 April 2010 - 04:06 AM #41

Tried without any cache. Not that bad. I did notice that it does take longer in categories like 3X. Put DB cache back and there is a definite improvement over no cache.

EDIT: I went back to file based cache and just moved cache dir to ramdisk. I hope 20MB is enough. In my experience it is fastest from what I have experienced today (including smartopt).

Please Test: cctvultra.com -- tell me if it is fast in your opinion (plz disregard the content, still messing around).
PM me today for design/coding projects :P

 
  • zeke
  • Megamind
  • Administrators
  • Join Date: 01-Nov 05
  • 472 posts

Posted 27 April 2010 - 11:38 AM #42

EDIT: I went back to file based cache and just moved cache dir to ramdisk. I hope 20MB is enough. In my experience it is fastest from what I have experienced today (including smartopt).


Have you tried db cache with ramdisk? I think It should work faster than file cache.

 
  • TexasGuy
  • Senior Member
  • Members
  • Join Date: 18-Mar 10
  • 546 posts

Posted 27 April 2010 - 12:21 PM #43

I thought about it but logically if you think about it, if you add a DB to the mix, you have an extra layer of I/O (talking about db engine). If you use files, at least it is natural to the server OS and requires no overhead. Since it is a ramdisk, fragmentation should not be a problem.

I'll test it with DB too.
PM me today for design/coding projects :P

 
  • adodric
  • Senior Member
  • Members
  • Join Date: 14-May 09
  • 320 posts

Posted 27 April 2010 - 02:06 PM #44

Sqlite is a flat file database. I've never used sqlite before and know very little about it, but I've read that it isn't good for multiple users or heavy load websites due to its lockout behavior on writes. Does this not affect the cs cart implementation as much because we are using it as a cache and there are more reads than writes? Even one write will lock out the entire database until completed, you can't even read from it, so it will affect all users.

One of the drawbacks of SQLite is that it is not really suitable in a situation where there are several users writing at the same time to the same database, because any write operation locks the entire database.

But SQLite will handle multi-users just fine if all they're doing is reading from the database, because reading does not require a lock. Therefore SQLite can run efficiently any website that doesn't require modifying records. Or a website where 1 person is doing the editing.


I know the majority of websites won't be affected by this, but I'd like to know Zeke's thoughts on this.

Someone posted a user load testing website a couple weeks ago (http://loadimpact.com/), perhaps running that on a cs cart with each of the cache methods and the no cache method would provide good data on which is best for multiple users on the site. If I get time I'll try to do it, if not do you think you could look into that sort of testing, TexasGuy?

 
  • TexasGuy
  • Senior Member
  • Members
  • Join Date: 18-Mar 10
  • 546 posts

Posted 27 April 2010 - 03:01 PM #45

@adodric - I would, if I will find time. However, it would still not show 100% if one is faster over another. Here we are not looking at a definite winner because we do not know which one, the difference is just too slim. I am afraid that the testing will not be exact because of different server setups that people have. Some are on shared, some are on dedicated, some servers have more file cache, others database cache, and so on... My point is that when dealing with such slim improvements, they may vary depending on a particular server setup. I don't think that a particular caching method will be a clear winner with the hard to detect improvements as they are now.


@zeke - I don't get why config.php has 2 lines for cache that are independent of each other:

define('DIR_CACHE', DIR_ROOT . '/var/cache/');
and
$config['cache_path'] = $config['http_path'] . '/var/cache';

Problem is, I have not seen the 2nd one and my admin JS was pointing to non existing JS files (I RENAMED CACHE FOLDER and I did not update the 2nd line which was pointing to the old cache folder).

Should $config['cache_path'] be something like = DIR_CACHE ?
PM me today for design/coding projects :P

 
  • zeke
  • Megamind
  • Administrators
  • Join Date: 01-Nov 05
  • 472 posts

Posted 28 April 2010 - 07:02 AM #46

I thought about it but logically if you think about it, if you add a DB to the mix, you have an extra layer of I/O (talking about db engine). If you use files, at least it is natural to the server OS and requires no overhead. Since it is a ramdisk, fragmentation should not be a problem.

I'll test it with DB too.


PHP is the bottleneck here, as it slows down when including a lot of files.

define('DIR_CACHE', DIR_ROOT . '/var/cache/');
and
$config['cache_path'] = $config['http_path'] . '/var/cache';


cache_path is used to access cached content via URL (google sitemap, pricelist addons and experimental javascript compression kit use it)

 
  • Traveler
  • Senior Member
  • Members
  • Join Date: 02-Feb 07
  • 920 posts

Posted 01 May 2010 - 02:45 PM #47

Zeke,

After downloading your new file I had a problem with caching please see this thread:
http://forum.cs-cart...82980#post82980
Is it possible that this is due to your file?

Edit:

I reverted back to the old file and the caching problem went away.

It seems that there is a bug in your file.

Version 4.9.2


 
  • Hetha
  • Junior Member
  • Members
  • Join Date: 08-Nov 09
  • 22 posts

Posted 03 May 2010 - 12:08 PM #48

We have had big problems where customers were getting stuck with "please be patient" when moving from the site to the payment processor. We were also finding it was taking customers up to 10 minutes to complete the payment stage (from order creation to order marked as processing)

We have a cron job set to clear cache - index.php?cc on an hourly basis but this only had marginal success. We were still having to clear cache manually with the cache folder containing up to 5000 files after 3 hours.

Searching for a solution yesterday i can across this thread downloaded and installed the new cache file.

So far it looks good, we have not cleared payment cache at all today, we have had sales and the times taken to complete payment stage are considerably faster.

One happy shop owner :-D

 
  • Shr3k
  • Junior Member
  • Members
  • Join Date: 25-Mar 08
  • 12 posts

Posted 04 May 2010 - 12:38 PM #49

Zeke,

After downloading your new file I had a problem with caching please see this thread:
http://forum.cs-cart...82980#post82980
Is it possible that this is due to your file?

Same for me.

 
  • Shr3k
  • Junior Member
  • Members
  • Join Date: 25-Mar 08
  • 12 posts

Posted 06 May 2010 - 07:31 PM #50

However this sqlite cache mod doesn't work for me. Still become instability and strange issues in checkout. I returned to original class.registry.php and to fn.catalog.php (CS-Cart support modified - with some filter caching disabled).

 
  • Ceceps
  • Junior Member
  • Members
  • Join Date: 01-May 10
  • 1 posts

Posted 06 May 2010 - 09:41 PM #51

I see my phpinfo() information. Result is my hosting disable PDO. I cannot use Zeke file. Is there another solutions for improve caching

 
  • Shr3k
  • Junior Member
  • Members
  • Join Date: 25-Mar 08
  • 12 posts

Posted 10 May 2010 - 12:16 PM #52

I see my phpinfo() information. Result is my hosting disable PDO. I cannot use Zeke file. Is there another solutions for improve caching


Try to comment out Registry::set($key, array($filters, $view_all)); line 2150 in file fn.catalog.php. This disables pfilter PHP arrays creating in var/cache.

 
  • nedd
  • Senior Member
  • Members
  • Join Date: 13-Jan 08
  • 125 posts

Posted 13 August 2010 - 06:35 PM #53

Zeke,

what happened with this solution? I had implemented in 2.0.15 and use it for a while. Now, after upgrade to 2.1.0, it's overwriten.

 

Posted 23 August 2010 - 01:56 PM #54

Will this still work with 2.1.0? A site we built with a lot of filters just kept dying as the cache got bigger until we implemented this solution.

Ideally this should be part of the core cs-cart, possibly with a check for minimum server requirements and a fallback to the standard caching system if the server doesn't meet requirements on install. Maybe you could select which option you want to use in the config.

 
  • Dezigner
  • Member
  • Members
  • Join Date: 12-Oct 09
  • 136 posts

Posted 23 August 2010 - 03:10 PM #55

Will this still work with 2.1.0? A site we built with a lot of filters just kept dying as the cache got bigger until we implemented this solution.

Ideally this should be part of the core cs-cart, possibly with a check for minimum server requirements and a fallback to the standard caching system if the server doesn't meet requirements on install. Maybe you could select which option you want to use in the config.


It's working on 2.1, for faster csc I'm using this mod and smartoptimizer... works great for me..

 
  • zeke
  • Megamind
  • Administrators
  • Join Date: 01-Nov 05
  • 472 posts

Posted 24 August 2010 - 01:12 PM #56

We've added support for several cache data storages in 2.1.1. At this moment the following methods are supported: file (default), sqlite, mysql and shared memory.

 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 11770 posts

Posted 31 August 2010 - 10:35 PM #57

Note that the biggest issue with the file caching method is the sheer number of files generated when using filters and features. A small to moderately sized site (2000 items) can generate 1000's of cache files (or db rows) for the filters and features associated with those products. Reading any directory (ram disk or file system based) will have performance implications due to the OS having to read the whole directory to find what it;s looking for.

The PDO method optimizes this somewhat, but the sheer number of entries makes the strategy less than performant.

Note also that when the cart tries to flush all these cached entires to disk that another process may try to read the cache file before the system has completed the update to the file. This is what causes the "unexpected $end" errors and other related problems with the file based registry files. The PDO version prevents this by providing its own internal locking of records.

EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • tbirnseth
  • CS Cart Expert
  • Authorized Reseller
  • Join Date: 08-Nov 08
  • 11770 posts

Posted 31 August 2010 - 10:40 PM #58

@adodric

Even one write will lock out the entire database until completed, you can't even read from it, so it will affect all users.

You actually want this behavior on a cache. Remember, this is a registry cache. In theory it should be "slow changing" data (data worth caching). Otherwise the data should not be cached if it is "fast changing". I question caching filter/feature info at all.... A separate table for "compiled filters and features" would be more optimal than adding this to the Registry.

EZ Merchant Solutions: Custom (USA based) B2B Development, Consulting, Development and Special Projects (get a quote here).
Commercial addons, payment methods and modifications to meet your business and operations needs.


 
  • Consultant
  • Member
  • Members
  • Join Date: 15-May 10
  • 170 posts

Posted 02 September 2010 - 02:46 PM #59

Tried without any cache. Not that bad. I did notice that it does take longer in categories like 3X. Put DB cache back and there is a definite improvement over no cache.

EDIT: I went back to file based cache and just moved cache dir to ramdisk. I hope 20MB is enough. In my experience it is fastest from what I have experienced today (including smartopt).

Please Test: cctvultra.com -- tell me if it is fast in your opinion (plz disregard the content, still messing around).


We're getting ready to launch our first CS-Cart site and it appears the cache was not designed very well and improvements are coming in 2.1.1. In the meantime however, I'm curious why there isn't discussion of the graph in this post (especially from Texas Guy who above claims the cache doesn't slow things down):

http://forum.cs-cart...25&postcount=38

It clearly shows page loading is faster without cache although there isn't specific details of the methodology used to arrive at the numbers used in the graph.

We followed the instructions to disable cache posted earlier. It breaks the class.registry.php file. Notice in the example there are two closing brackets commented out but there are not two matching opening brackets at the beginning commented out. We are running 2.0.14. And there is also three lines of code in there not shown in the disable cache instructions. Maybe if someone can clarify the code changes to disable cache. For now, we'll just have a cron even run every hour to erase all the files in the cache.

ALSO: We installed Zeke's SqlLite solution, worked fine, but it slows down the one-page checkout to the point it is unusable. We've found the one-page checkout is fastest right after clearing the cache!

Ou class.registry.php file with the commenting to disable looks like this (and doesn't work):

	private static function _set_cache($name, $data, $tables, $cache_level = NULL)
{
$fname = $name . '.' . $cache_level . '.php';
if (!is_dir(DIR_CACHE)) {
fn_mkdir(DIR_CACHE);
}

$change_cache = false;

if (self::$_cached_keys[$name]['track'] == true && !empty(self::$_cached_keys[$name . '_real_value']) && self::$_cached_keys[$name . '_real_value'] != $data) {
$change_cache = true;
}

// ** NEXT THREE LINES NOT IN THE DISABLE CACHE INFO ON CS-CART FORUM POST

// $db = self::init_db();
// $res = $db->query("SELECT NAME FROM cache WHERE name = '$fname'", PDO::FETCH_COLUMN, 0);
// $num_rows = ($fe = $res->fetch()) ? true : false;

// if (!empty($data) && (!$num_rows || $change_cache == true)) {

// if ($change_cache == true || $num_rows) {
// $db->query("DELETE FROM cache WHERE name = '$fname'");
// }

// $db->query("INSERT INTO cache (name, data) VALUES ('$fname', " . $db->quote(serialize($data)) . ")");
// //fn_print_r($fname, $db->errorInfo());
// /*fn_put_contents(DIR_CACHE . $fname, "<?php\n if ( !defined('AREA') ) { die('Access denied'); }\n \$_cache_data = " . var_export($data, true) . "\n?>");

// if (file_exists(DIR_CACHE . 'cache_update_handlers.php')) {
// include(DIR_CACHE . 'cache_update_handlers.php');
// }*/

// $res = $db->query("SELECT data FROM cache WHERE name = 'cache_handlers'", PDO::FETCH_COLUMN, 0);
// $cache_handlers = ($fe = $res->fetch()) ? unserialize($fe) : array();

// foreach ($tables as $table) {
// if (empty($cache_handlers[$table])) {
// $cache_handlers[$table] = array();
// }

// $cache_handlers[$table][$name] = true;
// }

// $db->query("INSERT INTO cache (name, data) VALUES ('cache_handlers', " . $db->quote(serialize($cache_handlers)) . ")");

// *** COMMENTING OUT CLOSING BRACKETS WITHOUT COMMENTING OUT CORRESPONDING OPENING BRACKETS?

// }
// }

//
// Retirieve data from the file cache
//
private static function _get_cache($name, $cache_level = NULL)
{


 
  • badmaash
  • Senior Member
  • Members
  • Join Date: 25-Dec 09
  • 456 posts

Posted 02 September 2010 - 06:24 PM #60

Note that the biggest issue with the file caching method is the sheer number of files generated when using filters and features. A small to moderately sized site (2000 items) can generate 1000's of cache files (or db rows) for the filters and features associated with those products. Reading any directory (ram disk or file system based) will have performance implications due to the OS having to read the whole directory to find what it;s looking for.

The PDO method optimizes this somewhat, but the sheer number of entries makes the strategy less than performant.

Note also that when the cart tries to flush all these cached entires to disk that another process may try to read the cache file before the system has completed the update to the file. This is what causes the "unexpected $end" errors and other related problems with the file based registry files. The PDO version prevents this by providing its own internal locking of records.


Hi there

You seem to understand the problem about using features and this used to kill cs-cart completely but in version 2.0.15 they fixed this problem..... my issues now is no longer on the front end but on the the checkout page and adding products. What happens is when you add a product the page just hangs even though the product really has been added..... the same goes for the checkout process..... the payment is taken but the user is just left sitting at the final page and the screen does not change.....

Can you recommend somthing for the mean time that might help in this issue or even anyone else?

Many Thanks

B