Since yesterday my website is having a very high server load on the cpanel. This is causing the entire website to slow down and the CMS is virtually inaccessible. I am on VPS server and before yesterday there were no load issues. But now the hosting company says the problem is with the Index file as it is consuming way too many resources. I contacted my developer but he says the index file is fine. I must mention that in the last week twice my databases crashed causing the website to show the error “Error Occurred”. Both times i was told that 2 tables namely CS Cart sessions and CS Cart SEO Names had gone corrupt and had to be repaired and optimized. But when i got them repaired yesterday since then the server load has been crazy. It is usually 1 or below but in the past 24 hours its constantly over 10 and memory usage is 70-90pc causing the website to not load at all or timeout.
Can someone help me out here? Is upgrading to CS-Cart 2.26 or CS Cart 3 gonna help?
Here are the logs sent to me by Host which is a VPS package btw when i issued a ticket
[size=4][font=Candara, Verdana, Arial, Helvetica]I have monitored your server for half an hour and found that the following :::[/font]
[font=Candara, Verdana, Arial, Helvetica]1) Initial Server load was :::[/font]
[font=Candara, Verdana, Arial, Helvetica]========================[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [~]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:30:52 up 5 days, 17:09, 0 users, load average: 15.24, 15.27, 15.80[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [~]#[/font]
[font=Candara, Verdana, Arial, Helvetica]========================[/font]
[font=Candara, Verdana, Arial, Helvetica]2) Found who is monopolizing or eating the CPU/Consuming CPU ::[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]%CPU PID USER COMMAND[/font]
[font=Candara, Verdana, Arial, Helvetica]9.5 22357 xxxx /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]8.8 22085 xxxx/usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]8.5 21834 xxxx /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]7.8 21611 xxxx /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]5.2 22353 xxxx/usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]5.0 22355 xxxx /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]0.00 Mb COMMAND[/font]
[font=Candara, Verdana, Arial, Helvetica]47.18 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]47.18 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]47.18 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]47.18 Mb /usr/bin/php /home/vxxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]47.18 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]46.93 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]18.62 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]18.37 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]15.10 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]11.77 Mb /usr/bin/php /home/xxxx/public_html/[/font][font=Candara, Verdana, Arial, Helvetica]index.php[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]From the above it seems that index.php file creating load on the server.[/font]
[font=Candara, Verdana, Arial, Helvetica]Considering this, I have done the following test ::[/font]
[font=Candara, Verdana, Arial, Helvetica]1) I have renamed the index.php file to index.php_bk and found that suddenly server load was decreasing ::[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:51:57 up 5 days, 17:30, 0 users, load average: 7.40, 13.53, 14.89[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:52:07 up 5 days, 17:31, 0 users, load average: 6.64, 13.16, 14.76[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:52:35 up 5 days, 17:31, 0 users, load average: 5.23, 12.19, 14.39[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:52:35 up 5 days, 17:31, 0 users, load average: 5.23, 12.19, 14.39[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]#[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]2) Then again I have revert back the changes, i.e. renamed index.php_bk to index.php and found that suddenly load was started increasing ::[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:54:39 up 5 days, 17:33, 0 users, load average: 3.67, 9.27, 13.05[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx[/home/xxxx/public_html]#w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:56:06 up 5 days, 17:35, 0 users, load average: 4.32, 7.96, 12.22[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:56:28 up 5 days, 17:35, 0 users, load average: 5.79, 8.06, 12.16[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]17:57:04 up 5 days, 17:36, 0 users, load average: 7.30, 8.20, 12.06[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]#[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]Now current load on the server is as below :::[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx [/home/xxxx/public_html]# w[/font]
[font=Candara, Verdana, Arial, Helvetica]18:08:26 up 5 days, 17:47, 1 user, load average: 12.87, 14.16, 13.98[/font]
[font=Candara, Verdana, Arial, Helvetica]USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT[/font]
[font=Candara, Verdana, Arial, Helvetica]root pts/1 182.177.189.216 18:04 3:31 0.04s 0.04s -bash[/font]
[font=Candara, Verdana, Arial, Helvetica]root@vpsxxxx[/home/xxxx/public_html]#[/font]
[font=Candara, Verdana, Arial, Helvetica]==============================[/font][font=Candara, Verdana, Arial, Helvetica]==================[/font]
[font=Candara, Verdana, Arial, Helvetica]I would request you that please contact to your web-developer in this regards and optimize the index.php file.[/font][/size]
Bump? can anyone here help?
Verify Cpanel > Latest visitors to schedule problems
Which version of CS-Cart are you using?
[quote name='Adrian8' timestamp='1352372494' post='148642']
Which version of CS-Cart are you using?
[/quote]
2.24 But this problem only happened yesterday. I have been on this version for more than 6 months without a major glitch
I know cs-cart sessions tables can get quite large in the database. There is the following in knowledge base (php application to repair):-
Crashed CS-Cart Sessions table
What does a 'netstat' tell you about active tcp connections on port 80 at the time your site is heavily loaded?
My bet is that you are simply getting hammered by a caching proxy server or similar crawler that wants to follow all your links. You will probably find that you have many connections from the same IP address. Temporarily block that IP and your load will go down dramatically. You can then research the IP to determine if it is valid or not. If it's related to a .us.gov' or '.mil' site then pretty good bet it's a caching proxy server. If you have multiple languages and a crawler just reads as fast as it can, then you will logarithmically increase your load by the number of languages.
[quote name='tbirnseth' timestamp='1352435641' post='148745']
What does a 'netstat' tell you about active tcp connections on port 80 at the time your site is heavily loaded?
My bet is that you are simply getting hammered by a caching proxy server or similar crawler that wants to follow all your links. You will probably find that you have many connections from the same IP address. Temporarily block that IP and your load will go down dramatically. You can then research the IP to determine if it is valid or not. If it's related to a .us.gov' or '.mil' site then pretty good bet it's a caching proxy server. If you have multiple languages and a crawler just reads as fast as it can, then you will logarithmically increase your load by the number of languages.
[/quote]
no thats not the issue. already got that checked. Every now and then the site goes down with the error “Error Occurred”
I am sick and tired of CS-Cart. Thinking of dumping it altogether. Problem is i hv invested a lot in SEO and i had just started to get results.
Everytime i contact my host they tell my index file is consuming too many resources. I did not do any change to the index file recently.
Have a look at it
[i]/***************************************************************************[/i]
[i]* *[/i]
[i]* Copyright (c) 2004 Simbirsk Technologies Ltd. All rights reserved. *[/i]
[i]* *[/i]
[i]* This is commercial software, only users who have purchased a valid *[/i]
[i]* license and accept to the terms of the License Agreement can install *[/i]
[i]* and use this program. *[/i]
[i]* *[/i]
[i]****************************************************************************[/i]
[i]* PLEASE READ THE FULL TEXT OF THE SOFTWARE LICENSE AGREEMENT IN THE *[/i]
[i]* "copyright.txt" FILE PROVIDED WITH THIS DISTRIBUTION PACKAGE. *[/i]
[i]****************************************************************************/[/i]
[i]//[/i]
[i]// $Id: index.php 11039 2010-10-27 12:43:28Z 2tl $[/i]
[i]//[/i]
[i]define('AREA', 'C');[/i]
[i]define('AREA_NAME', 'customer');[/i]
[i]define('ACCOUNT_TYPE', 'customer');[/i]
[i]require dirname(__FILE__) . '/prepare.php';[/i]
[i]require dirname(__FILE__) . '/init.php';[/i]
[i]define('INDEX_SCRIPT', Registry::get('config.customer_index'));[/i]
[i]fn_dispatch();[/i]
[i]?>
Tell me is there anything unusual in this index file? Something which is causing the load?
index.php is just the gateway and not necessarily the script that is causing your issue. Your file does look a little different than the original 2.2.4 index.php but I doubt it is your issue since it just occurred all of a sudden.
If you have the statistics addon installed then uninstall it (and leave it uninstalled) and do a repair on the data base.
```php
/***************************************************************************
-
*
* *
* This is commercial software, only users who have purchased a valid *
* license and accept to the terms of the License Agreement can install *
* and use this program. *
* *
****************************************************************************
* PLEASE READ THE FULL TEXT OF THE SOFTWARE LICENSE AGREEMENT IN THE *
* "copyright.txt" FILE PROVIDED WITH THIS DISTRIBUTION PACKAGE. *
****************************************************************************/
//
// $Id$
//
define('AREA', 'C');
define('AREA_NAME', 'customer');
require dirname(__FILE__) . '/prepare.php';
require dirname(__FILE__) . '/init.php';
define('INDEX_SCRIPT', Registry::get('config.customer_index'));
fn_dispatch();
?> ```
If you are getting the “Error Occurred” message, then put your store in Devlopment mode or turn on logging for database errors and see what the errors are.
My experience is that if you have a VPS then it is under-configured. If you are on a shared server then you should probably move to a VPS.
My experience also indicates that the number of active connections into your site is probably higher than your environment can sustain.
As mentioned above if your VPS has not been optimally configured then it is quite possible for all memory and swap resources to be consumed when under load. Once all resources are consumed on the VPS then it could become unstable, causing database server crash (which may result in corrupt database) and ultimately server resets.
What spec VPS are you using?
Have checked the settings for prefork or worker MPM (assuming Apache)
OK here is the thing. We tried a few things and it kind of worked but i want to know your input whether it really could be the problem or not:
- I disabled a Live Chat addon calld Zopim which also had a crawler
- Everytime the db error would come the hosts (Which are EuKhost and I am on a VPS btw) would give me 2 reasons
i) MySql got stuck
ii) 2 db tables, namely CS Cart sessions and CS Cart SEO Names have corrupted
So my developer set up a cron job which would automatically empty the CS Cart sessions table every hour and also would do a repair on the db if any of them got corrupt every hour.
My question to you guys is if the CS Cart sessions table had gotten too big then how come i didnt face this issue in the last 6 months and why now? As for my visitor count it is somewhere between 1000-1300 daily which im assuming is not a lot of traffic.
Please let me know if this could be the issue or am i missing something here. So far the website is more stable. Once or twice there were load spikes but nothing serious. As for my VPS configuration is concerned, its got 750mb RAM, 20GB space…
I was considering to purchase the package giving me 2048 RAm and 60GB space but as i said i have been running this package for 6 months or more without any major issue related to traffic.
I doubt your sessions table got “too big”. It would appear that mySQL crashed during access to those tables and therefore the tables were corrupted when mysqld restarted.
Clearing your sessions table will only make your customer's experience suffer. You will find things like carts disappearing, checkout process abandoning, etc. I think that's a very BAD idea. You are treating a symptom rather than a cause. What is the current connection count to port 80 (per netstat)? Not by IP, but by total active connections? My guess is your 750MB VPS is way to small to handle the load being placed on it.
We tried changing the setting of emptying sessions table to once a day but that resulted in the load returning in a few hours…nothing crashed but it came rather heavy at once…
about the VPS well as i said i have faced a surge in traffic in the last few months…my usual traffic used to be between 600-800 but after investing in SEO it rose to 1000-1300 unique visitors daily…but this surge has been since September now…if the VPS was not equipped to handle this traffic it should have broken down 2 months ago why now?
You will of course have unique visitors + web crawlers, depending on when these occur you could quite easily use up available resources. Have you checked all logs for any clues, mySQL restart, server restarts etc.
You should consider a higher spec VPS (more memory + cpu resources), and/or either use a lighter webserver instead of Apache (but this is a little more involved to setup).
So all you guys think i should move to a better VPS package and remove the cron job for clearing CS Cart Sessions table?
Well ok il move to a new server and update you guys whether it has solved the problem or not.
Also can anyone tell me how many users can cs-cart handle in a suitable environment. Like if a VPS has say :
2 Core
2GB RAM
60GB Disk Space
500GB Bandwidth.
How many visitors daily or per month such a server can handle on CS-Cart?
I need rough estimates ofcourse.
The answer unfortunately is “it depends”. It depends on many factors. How many products/categories, how many product features, use of product filters, addons, etc. There is no pat answer.
I have clients that average 1200 - 2000 unique visitors a day quite easily on a VPS similar to what you describe above. I also have a client that runs 600/day and he continually has problems do to caching servers hammering his site. (usually *.us.gov' sites.). He has to block those IP's (and those customers) to keep things alive. But that's a function of what he sells and whom he sells to. I.e. specialty auto parts and most of his customers are in the 18-30 demographic and many are military).
Also note that VPS is still a shared server (shared hardware) and depending on the integrity of your hosting provider you may be on an already overloaded server and/or they may have implemented your vps in a cloud which will be highly variable.
No easy answers and it all takes time and research and experimentation. But zapping the session file is hitting the symptom rather than the cause.
A typical browser may open as many 10 connections or more, if using Apache with prefork then you will require upto 10 processes to server requests. If you have not set up browser caching for files such as css, js and images, then each page request may require in the order of 50 requests (depends on graphics, etc). So first you should check you have setup browser caching as this will reduce the load on the server and give faster improved experience for your customers.
You could also use a lightweight server such as NGinx, then you could reduce the amount of memory required (all static files will be handled with very little overhead). You could also use some advanced features to rate reduce number of requests, so you could limit page requests to 2/second from same address (which will help against some of those nuisance proxies).
Even with a lightweight server such as NGinx, you may still require more CPU and memory (if you have many concurrent users and also size/complexity features use of the store).
If you want to try and assess loads and concurrent users you could try a service such as
[url=“http://loadimpact.com/”]http://loadimpact.com/[/url]
You might want to check some of your addons. With thanks to Jesse, I learned that even if you do an upgrade, the addons are brought forward. So it is possible that you could have an addon that is conflicting with something.
I had a similar problem with a site and had Jesse take a look at it. He was able to get everything fixed up and running great. If you aren't able to get it, I'd highly recommend talking to him.
Thanks,
Brandon