v2.1.4 caused higher cpu usage and memory consumption

[B]**updated 15 May 2011:

I’ve changed the connection type to mysql instead of using mysqli. And things are much much more stable now in terms of cpu and memory usage.

  • Default cs-cart configuration is “mysql” (found in config.local.php)
  • Note: If you are using MySQL versions 4.1.3 or later it is strongly recommended that you use “mysqli” extension.
  • If you are using versions of MySQL lower than 4.1.3, and is experiencing high cpu usage/memory consumption after change to “mysqli”, stick to the default “mysql”

    **[/B]





    I’ve just upgrade my site to v2.1.4 from v2.1.2 yesterday.

    I backup the database and upgrade them locally, Before exporting the database and re-importing them into a fresh v2.1.4 install on the server to upgrade.



    The upgrade was fine. My site was transit into the new v2.1.4 installed directory and all.



    But one day later, while I was out the whole day, I came home and checked that my site was down.



    Ssh in, and saw that about 1600+ core.xxx core dumps file was generated. And I got a error.log file that was 5mb in size.



    Open the error.log file and saw that most of the errors were:


[14-Mar-2011 15:44:23] PHP Warning: mysqli_error() expects parameter 1 to be mysqli, boolean given in /home/user/public_html/user/core/db/mysqli.php on line 149
[14-Mar-2011 15:44:23] PHP Warning: mysqli_errno() expects parameter 1 to be mysqli, boolean given in /home/user/public_html/user/core/db/mysqli.php on line 140
[14-Mar-2011 15:44:23] PHP Warning: mysqli_query() expects parameter 1 to be mysqli, boolean given in /home/user/public_html/user/core/db/mysqli.php on line 71


and occasionally this error will appear in midst of the lines shown above:

```php

[14-Mar-2011 15:44:50] PHP Warning: Cannot modify header information - headers already sent by (output started at /home/user/public_html/user/core/db/mysqli.php:44) in /home/user/public_html/user/core/fn.control.php on line 490

[/core]



Finally tapering off with repeated lines of:


[14-Mar-2011 03:49:15] PHP Warning: mysqli::mysqli() [mysqli.mysqli]: (HY000/2002): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) in /home/user/public_html/user/core/db/mysqli.php on line 44
[14-Mar-2011 03:49:15] PHP Warning: mysqli_query() [function.mysqli-query]: Couldn't fetch cmysqli in /home/user/public_html/user/core/db/mysqli.php on line 71
[14-Mar-2011 03:49:15] PHP Warning: mysqli_errno() [function.mysqli-errno]: Couldn't fetch cmysqli in /home/user/public_html/user/core/db/mysqli.php on line 140
[14-Mar-2011 03:49:15] PHP Warning: mysqli_errno() [function.mysqli-errno]: Couldn't fetch cmysqli in /home/user/public_html/user/core/db/mysqli.php on line 140
[14-Mar-2011 03:49:15] PHP Warning: mysqli_error() [function.mysqli-error]: Couldn't fetch cmysqli in /home/user/public_html/user/core/db/mysqli.php on line 149
[14-Mar-2011 03:49:15] PHP Warning: mysqli_errno() [function.mysqli-errno]: Couldn't fetch cmysqli in /home/user/public_html/user/core/db/mysqli.php on line 140
[14-Mar-2011 04:03:04] PHP Warning: mysqli::mysqli() [mysqli.mysqli]: (42000/1203): User user already has more than 'max_user_connections' active connections in /home/user/public_html/user/core/db/mysqli.php on line 44




My webhost (shared hosting on DTH) also informed me that the site’s index file was using lot of CPU and memory. The cpu load spiked to 260+ or more.



Looking from the logs, it seemed like the mysql server was overloaded with cs-cart queries.



Is there a way to gracefully catch such mysql exception? So that it will not give 500 error, and show a custom error page where I can state “Our site is having some server issues and will be back soon”



Although v2.1.2 occasionally have similar problems, but it was not as bad as v2.1.4 where in just 1 day, such problems starts to occur.



I’ve change the sql type to mysql for now (from mysqli) and see if the same problem will still occur,.

And change the memory limit from 128M to 64M in config.local.php file too.

Do you think it will help?



And why is the script generating these core dumps. Webhost support just said that the script application got corrupted. But that is not enough information for me to find out what exactly is causing these problems.



My site have statistics turn off, about 8500+ products, 30 plus categories/sub-categories, 4 product filters with 4-9 fitlers each.



On a sidenote, though not exactly related to cpu and memory usage. Does anyone have this issue where if you delete more than for eg. 40products or so at the same time, the admin page will redirect to [url]Website Hosting - Mysite.com page? I wonder why is this happening.



Will appreciate anyone with the technical knowledge to enlighten me here. Thanks!

I have the same problem. v2.1.2

i have the same problem with v2.1.4



only 1500 products

This is a server configuration issue if you are using vanilla CS-Cart files.

user/core/db/mysqli.php refers to SQL Enchanced / SQLite not being called correctly (refers to headers already being sent).



I suggest you replace the entire /core/ directory from a vanilla copy and see how it goes from there.



J.

I get the same warnings from time to time but I haven’t noticed any CPU overage. It seems to happen only during exim. CS’s fix to this is to switch to mysql instead. LOL

My server does not work with mysqli either

I´ve not such experiences because i havent actually run a shop. But whole the time since i know cs-cart, the software has performance problems at the frontend.

i have the same problem right now…is there any way to reduce the amount of memory cs cart uses?



I only have 10 products…

So is there really enough reason to use “mysqli” when it bogs down the server? Does it really speed up or make anything better? From this post and a few others it sounds like the “mysql” (without the "i), would be better.

I run and maintain multiple cs-cart sites (8 as of current), each with several hundred products (typically 500+) on 1 fairly beefy server, and we see very little slowdowns or issues with either system memory (ram) or CPU usage problems.



CS-Cart has always been very fast for us, since version 1.3.5 all the way up to the current versions we run, which is currently a mix of 2.0.14 and 2.1.4, as get all sites upgraded to 2.1.4, 2.1.4 is also the fastest, since block caching now exists.



We also utilize executing PHP via Fast CGI (in daemon mode (mod_fcgid)), so that it remains in memory and just be ran again, versus having to be re-parsed, op compiled and then re-executed.

So are you using “mysqli” or the normal, “mysql”?

[quote name=‘clips’]So are you using “mysqli” or the normal, “mysql”?[/quote]



Sorry, I totally meant to mention that! Duh! We are using the “enhanced” mysqli type.

From the PHP manual…



What is PHP’s mysqli Extension?



The mysqli extension, or as it is sometimes known, the MySQL improved extension, was developed to take advantage of new features found in MySQL systems versions 4.1.3 and newer. The mysqli extension is included with PHP versions 5 and later.



The mysqli extension has a number of benefits, the key enhancements over the mysql extension being:



Object-oriented interface



Support for Prepared Statements



Support for Multiple Statements



Support for Transactions



Enhanced debugging capabilities



Embedded server support



Note: If you are using MySQL versions 4.1.3 or later it is strongly recommended that you use this extension.



As well as the object-oriented interface the extension also provides a procedural interface.

tbirnseth’s reply which states “Note: If you are using MySQL versions 4.1.3 or later it is strongly recommended that you use this extension.” is good advice. I think I will add that to the updated note on my first post for others to refer.



Though if you are on lower than MySQL versions 4.1.3, then you should stick to mysql.



My change from “mysqli” to “mysql” is the one single change that dramatically bring down the cpu/memory usage for my site. And yes, my shared server running this site, unfortunately, is still on mysql version lower than 4.1.3, 4.1.22 to be exact.



Currently, it is stable and running fine with no spikes in cpu/memory loads whatsoever caused by cscart

My bet is that in bringing down the cpu/memory useage you have increased both the file I/O and the amount of time it takes to make your queries.



People seem to think there’s some majic cookie to increasing performance. It is ALWAYS a trade off. You trade a resource you have in excess to improve one that is struggling. It’s like the law of physics. If you have the CPU and memory available, why not use it?

[quote name=‘tbirnseth’]My bet is that in bringing down the cpu/memory useage you have increased both the file I/O and the amount of time it takes to make your queries.



People seem to think there’s some majic cookie to increasing performance. It is ALWAYS a trade off. You trade a resource you have in excess to improve one that is struggling. It’s like the law of physics. If you have the CPU and memory available, why not use it?[/QUOTE]



Most probably you are correct and of course if there is the CPU and memory available, using it will be the best solution.

However, using “mysqli” for the site on my shared server currently, will spike cpu loads up to over 100-200+ and hang the mysql server, about 4-5 times a day. So the trade off for stability is better for my case.

Then you should talk to your hosting company about the underlying configuration of mysql. Something is wrong if that’s occuring. Either there are not enough threads to handle the load or the buffer sizes are too small.

[quote name=‘tbirnseth’]Then you should talk to your hosting company about the underlying configuration of mysql. Something is wrong if that’s occuring. Either there are not enough threads to handle the load or the buffer sizes are too small.[/QUOTE]



yeap I shall. thanks for the advice! ^^

[quote name=‘tbirnseth’]…

If you have the CPU and memory available, why not use it?[/QUOTE]



So what CPU and memory do all the veterans and pros suggest that someone have? I know it depends on products, etc. But since CS seems to still be a resource pig, what amount of resources should you have at a minimum?

Unless you’re on a dedicated server you can’t control it. And no, a VPS does not DEDICATE resources, it ALLOCATES. I will guarantee you that any server supporting VPS’s has allocated VPS resources that exceed the physical resources (other than disk) of the underlying server.



My statement was more in regards to “why worry about cpu usage and memory?”. I.e. why use mysql versus myslqi to reduce cpu/memory usage while probably increasing the demand on much slower resources such as file I/O.