Jump to content

  • You cannot start a new topic
  • You cannot reply to this topic

Urgent Help Needed - Too Many Connections - Site Mostly Down Rate Topic   - - - - -

 
  • Flow
  • Super Duper and Amazingly Sexy Senior
  • Members
  • Join Date: 13-Oct 10
  • 2307 posts

Posted 16 March 2019 - 06:44 AM #1

Hi! Unfortunately we are facing a really strange issue.

 

Suddenly since yesterday morning, cs-carts log is saying User xxx already has more than 'max_user_connections' active connections in /home/xxx/public_html/app/Tygh/Backend/Database/Mysqli.php on line 40

 

Our host has tried all night to resolve it without success. They even moved the entire server to a new node and still the issue remains. I believe it's not a matter of tweaking the configuration a bit - the host already tried this and why would this suddenly be needed without any spike in visitors (we could easily handle 150, now we can't handle 2).

 

This same setup has run without any issue at all for months and it's even showing these errors with just 1 visitor. We can't sell anything or do any work. 

 

If there is anybody who might know something or could have a look I'd be very grateful.

 

Thanks!


When life hands you lemons, bring on the Tequila baby!

 
  • The Tool
  • Been Here Way Too Long Member
  • Members
  • Join Date: 30-Mar 07
  • 3848 posts

Posted 16 March 2019 - 12:24 PM #2

Is it a specific user or does it happen if you access the site?

 

It sounds like an issue I had where anonymous bots were running random search queries.


 
  • Flow
  • Super Duper and Amazingly Sexy Senior
  • Members
  • Join Date: 13-Oct 10
  • 2307 posts

Posted 16 March 2019 - 12:29 PM #3

Hi Tool, thanks for your reply. 

 

A 6th person at my host just finally found there was an unusually high number of syn connections. She tuned the firewall and now it runs OK again.

 

I already blocked most bots except for the important ones.

 

Why someone would attack us (unless it's a competitor?!) remains a mystery to me .. or could this have been something else?


When life hands you lemons, bring on the Tequila baby!

 
  • The Tool
  • Been Here Way Too Long Member
  • Members
  • Join Date: 30-Mar 07
  • 3848 posts

Posted 16 March 2019 - 12:33 PM #4

I never could figure who was responsible for the search queries.  They came from proxies all over the world.

 

Do you know what the requests being made were?


 
  • Flow
  • Super Duper and Amazingly Sexy Senior
  • Members
  • Join Date: 13-Oct 10
  • 2307 posts

Posted 16 March 2019 - 12:40 PM #5

Crazy. I don't know, I will ask my host if they can see it. 

 

I just checked WHM's daily log and can clearly see that suddenly our carts usage went from a couple of percent to 80 in cpu and similar spikes in mysql. Before that, our wordpress blog was actually eating up more resources than cs-cart!


When life hands you lemons, bring on the Tequila baby!

 
  • The Tool
  • Been Here Way Too Long Member
  • Members
  • Join Date: 30-Mar 07
  • 3848 posts

Posted 16 March 2019 - 12:46 PM #6

You would have to check the account specific visitor log.


 
  • Flow
  • Super Duper and Amazingly Sexy Senior
  • Members
  • Join Date: 13-Oct 10
  • 2307 posts

Posted 16 March 2019 - 01:57 PM #7

Hmm I don't see anything strange in there. It looks totally normal.


When life hands you lemons, bring on the Tequila baby!

 
  • The Tool
  • Been Here Way Too Long Member
  • Members
  • Join Date: 30-Mar 07
  • 3848 posts

Posted 16 March 2019 - 02:14 PM #8

Interesting.  I guess there is nothing to log since the connection is only half open.?


 
  • Flow
  • Super Duper and Amazingly Sexy Senior
  • Members
  • Join Date: 13-Oct 10
  • 2307 posts

Posted 16 March 2019 - 03:37 PM #9

Yeah could be, that's what I was thinking too.


When life hands you lemons, bring on the Tequila baby!

 
  • memel
  • Senior Member
  • Members
  • Join Date: 29-May 17
  • 216 posts

Posted 16 March 2019 - 08:11 PM #10

 

I already blocked most bots except for the important ones.

 

 

 

Could you share how to do that and which one (list) to avoid things happened?

 

Thank you in advance.


 
  • Flow
  • Super Duper and Amazingly Sexy Senior
  • Members
  • Join Date: 13-Oct 10
  • 2307 posts

Posted 18 March 2019 - 09:24 AM #11

Could you share how to do that and which one (list) to avoid things happened?

 

Sure, you can refuse them in your robots.txt file. Here is my robots.txt file

User-agent: *
Disallow: /app/
Disallow: /store_closed.html

# Disallow: Sistrix
User-agent: sistrix
Disallow: /

# Disallow: Sistrix
User-agent: SISTRIX Crawler
Disallow: /

# Disallow: Sistrix
User-agent: SISTRIX
Disallow: /

# Disallow: SEOkicks-Robot
User-agent: SEOkicks-Robot
Disallow: /

# Disallow: jobs.de-Robot
User-agent: jobs.de-Robot
Disallow: /

# Backlink Analysis
user-agent: AhrefsBot
disallow: /

# Bot der Leipziger Unister Holding GmbH
user-agent: UnisterBot
disallow: /

# http://moz.com/products
User-agent: dotbot
Disallow: /

# http://www.searchmetrics.com
User-agent: SearchmetricsBot
Disallow: /

# http://www.majestic12.co.uk/projects/dsearch/mj12bot.php
User-agent: MJ12bot
Disallow: /

# http://www.domaintools.com/webmasters/surveybot.php
User-agent: SurveyBot
Disallow: /

# http://www.seodiver.com/bot
user-agent: SEOdiver
disallow: /

# http://openlinkprofiler.org/bot
User-agent: spbot
Disallow: /

# http://www.wotbox.com/bot/
User-agent: wotbox
Disallow: /

# http://www.opensiteexplorer.org/dotbot
# http://moz.com/researchtools/ose/dotbot
User-agent: dotbot
Disallow: /

# http://www.meanpath.com/meanpathbot.html
User-agent: meanpathbot
Disallow: /

# http://www.backlinktest.com/crawler.html
User-agent: BacklinkCrawler 
Disallow: /

# http://www.brandwatch.com/magpie-crawler/
User-agent: magpie-crawler
Disallow: /

# http://filterdb.iss.net/crawler/
User-agent: oBot
Disallow: /

User-agent: fr-crawler
Disallow: /

# http://webmeup-crawler.com
User-agent: BLEXBot
Disallow: /

# https://megaindex.com/crawler
User-agent: MegaIndex.ru
Disallow: /

User-agent: megaindex.com
Disallow: /

# http://www.cloudservermarket.com
User-Agent: CloudServerMarketSpider
Disallow: /

# http://www.trendiction.de/de/publisher/bot
User-Agent: trendictionbot 
Disallow: /

# http://www.exalead.com
User-agent: Exabot
Disallow: /

# http://www.career-x.de/bot.html
User-agent: careerbot
Disallow: /

# https://www.lipperhey.com/en/about/
User-agent: Lipperhey-Kaus-Australis
Disallow: /

User-agent: seoscanners.net
Disallow: /

User-agent: MetaJobBot
Disallow: /

User-agent: Spiderbot
Disallow: /

User-agent: LinkStats
Disallow: /

User-agent: JobboerseBot
Disallow: /

User-agent: ICCrawler
Disallow: /

User-agent: Plista
Disallow: /

User-agent: Domain Re-Animator Bot
Disallow: /

# https://www.lipperhey.com/en/about/
User-agent: Lipperhey-Kaus-Australis
Disallow: /

# https://turnitin.com/robot/crawlerinfo.html
User-agent: turnitinbot
Disallow: /

# http://help.coccoc.com/
User-agent: coccoc
Disallow: /

# ubermetrics-technologies.com
User-agent: um-IC
Disallow: /

# datenbutler.de
User-agent: mindUpBot
Disallow: /

# http://searchgears.de/uber-uns/crawling-faq.html
User-agent: sg-Orbiter
Disallow: /

# http://commoncrawl.org/faq/
User-agent: CCBot
Disallow: /

# https://www.qwant.com/
User-agent: Qwantify
Disallow: /

# http://linkfluence.net/
User-agent: Kraken
Disallow: /

# http://www.botje.com/plukkie.htm
User-agent: plukkie 
Disallow: /

# https://www.safedns.com/searchbot
User-agent: SafeDNSBot
Disallow: /

# http://www.haosou.com/help/help_3_2.html
User-agent: 360Spider
Disallow: /

# http://www.haosou.com/help/help_3_2.html
User-agent: HaosouSpider
Disallow: /

# http://www.moz.com/dp/rogerbot
User-agent: rogerbot
Disallow: /

# http://www.openhose.org/bot.html
User-agent: OpenHoseBot
Disallow: /

# http://www.screamingfrog.co.uk/seo-spider/
User-agent: Screaming Frog SEO Spider
Disallow: /

# http://thumbsniper.com
User-agent: ThumbSniper
Disallow: /

# http://www.radian6.com/crawler
User-agent: R6_CommentReader
Disallow: /

User-agent: ImplisenseBot
Disallow: /

# http://cliqz.com/company/cliqzbot
User-agent: Cliqzbot
Disallow: /

# https://www.aihitdata.com/about
User-agent: aiHitBot
Disallow: /

# http://www.trendiction.com/en/publisher/bot
User-Agent: trendictionbot
Disallow: /

# http://warebay.com/bot.html
User-agent: WBSearchBot
Disallow: /

When life hands you lemons, bring on the Tequila baby!

 
  • Jacek
  • Senior Member
  • Members
  • Join Date: 13-Dec 12
  • 156 posts

Posted 18 March 2019 - 10:00 AM #12

If you're using nginx I would highly recommend to use this config. I had similar issue with many malicious bots and it worked like a charm:

https://github.com/m...bad-bot-blocker


 
  • johnbol1
  • Never Re
  • Members
  • Join Date: 23-Feb 10
  • 4667 posts

Posted 18 March 2019 - 11:59 AM #13

Isnt the robots text ignored by the badbots ?


Custom printed hi visibility clothing sale the UK's online hivis safety shop
v4.5.2

 
  • memel
  • Senior Member
  • Members
  • Join Date: 29-May 17
  • 216 posts

Posted 21 March 2019 - 07:01 AM #14

 

Sure, you can refuse them in your robots.txt file. Here is my robots.txt file

User-agent: *
Disallow: /app/
Disallow: /store_closed.html

# Disallow: Sistrix
User-agent: sistrix
Disallow: /

# Disallow: Sistrix
User-agent: SISTRIX Crawler
Disallow: /

# Disallow: Sistrix
User-agent: SISTRIX
Disallow: /

# Disallow: SEOkicks-Robot
User-agent: SEOkicks-Robot
Disallow: /

# Disallow: jobs.de-Robot
User-agent: jobs.de-Robot
Disallow: /

# Backlink Analysis
user-agent: AhrefsBot
disallow: /

# Bot der Leipziger Unister Holding GmbH
user-agent: UnisterBot
disallow: /

# http://moz.com/products
User-agent: dotbot
Disallow: /

# http://www.searchmetrics.com
User-agent: SearchmetricsBot
Disallow: /

# http://www.majestic12.co.uk/projects/dsearch/mj12bot.php
User-agent: MJ12bot
Disallow: /

# http://www.domaintools.com/webmasters/surveybot.php
User-agent: SurveyBot
Disallow: /

# http://www.seodiver.com/bot
user-agent: SEOdiver
disallow: /

# http://openlinkprofiler.org/bot
User-agent: spbot
Disallow: /

# http://www.wotbox.com/bot/
User-agent: wotbox
Disallow: /

# http://www.opensiteexplorer.org/dotbot
# http://moz.com/researchtools/ose/dotbot
User-agent: dotbot
Disallow: /

# http://www.meanpath.com/meanpathbot.html
User-agent: meanpathbot
Disallow: /

# http://www.backlinktest.com/crawler.html
User-agent: BacklinkCrawler 
Disallow: /

# http://www.brandwatch.com/magpie-crawler/
User-agent: magpie-crawler
Disallow: /

# http://filterdb.iss.net/crawler/
User-agent: oBot
Disallow: /

User-agent: fr-crawler
Disallow: /

# http://webmeup-crawler.com
User-agent: BLEXBot
Disallow: /

# https://megaindex.com/crawler
User-agent: MegaIndex.ru
Disallow: /

User-agent: megaindex.com
Disallow: /

# http://www.cloudservermarket.com
User-Agent: CloudServerMarketSpider
Disallow: /

# http://www.trendiction.de/de/publisher/bot
User-Agent: trendictionbot 
Disallow: /

# http://www.exalead.com
User-agent: Exabot
Disallow: /

# http://www.career-x.de/bot.html
User-agent: careerbot
Disallow: /

# https://www.lipperhey.com/en/about/
User-agent: Lipperhey-Kaus-Australis
Disallow: /

User-agent: seoscanners.net
Disallow: /

User-agent: MetaJobBot
Disallow: /

User-agent: Spiderbot
Disallow: /

User-agent: LinkStats
Disallow: /

User-agent: JobboerseBot
Disallow: /

User-agent: ICCrawler
Disallow: /

User-agent: Plista
Disallow: /

User-agent: Domain Re-Animator Bot
Disallow: /

# https://www.lipperhey.com/en/about/
User-agent: Lipperhey-Kaus-Australis
Disallow: /

# https://turnitin.com/robot/crawlerinfo.html
User-agent: turnitinbot
Disallow: /

# http://help.coccoc.com/
User-agent: coccoc
Disallow: /

# ubermetrics-technologies.com
User-agent: um-IC
Disallow: /

# datenbutler.de
User-agent: mindUpBot
Disallow: /

# http://searchgears.de/uber-uns/crawling-faq.html
User-agent: sg-Orbiter
Disallow: /

# http://commoncrawl.org/faq/
User-agent: CCBot
Disallow: /

# https://www.qwant.com/
User-agent: Qwantify
Disallow: /

# http://linkfluence.net/
User-agent: Kraken
Disallow: /

# http://www.botje.com/plukkie.htm
User-agent: plukkie 
Disallow: /

# https://www.safedns.com/searchbot
User-agent: SafeDNSBot
Disallow: /

# http://www.haosou.com/help/help_3_2.html
User-agent: 360Spider
Disallow: /

# http://www.haosou.com/help/help_3_2.html
User-agent: HaosouSpider
Disallow: /

# http://www.moz.com/dp/rogerbot
User-agent: rogerbot
Disallow: /

# http://www.openhose.org/bot.html
User-agent: OpenHoseBot
Disallow: /

# http://www.screamingfrog.co.uk/seo-spider/
User-agent: Screaming Frog SEO Spider
Disallow: /

# http://thumbsniper.com
User-agent: ThumbSniper
Disallow: /

# http://www.radian6.com/crawler
User-agent: R6_CommentReader
Disallow: /

User-agent: ImplisenseBot
Disallow: /

# http://cliqz.com/company/cliqzbot
User-agent: Cliqzbot
Disallow: /

# https://www.aihitdata.com/about
User-agent: aiHitBot
Disallow: /

# http://www.trendiction.com/en/publisher/bot
User-Agent: trendictionbot
Disallow: /

# http://warebay.com/bot.html
User-agent: WBSearchBot
Disallow: /

Thank you.


 
  • Flow
  • Super Duper and Amazingly Sexy Senior
  • Members
  • Join Date: 13-Oct 10
  • 2307 posts

Posted 21 March 2019 - 12:42 PM #15

Isnt the robots text ignored by the badbots ?

 

Maybe by the very bad bots, but the unneccesary bots seem to respect the robots.txt


When life hands you lemons, bring on the Tequila baby!

 
  • soft-solid
  • Junior Member
  • Members
  • Join Date: 19-Apr 10
  • 915 posts

Posted 12 April 2019 - 07:45 PM #16

Hello.

 

A better solution is to block robots in the .htaccess file.

 

Best regards

Robert.


Team of SoftSolid
cs-cart.pl
 
  • theanalyst
  • Newbie
  • Members
  • Join Date: 17-Oct 19
  • 12 posts

Posted 17 October 2019 - 03:43 PM #17

Hello.

 

A better solution is to block robots in the .htaccess file.

 

Best regards

Robert.

 

 

Agree with this


Back to Server configuration, Perfomance Optimizations
  1. CS-Cart Community Forums
  2. > Multi-Vendor
  3. > Server configuration, Perfomance Optimizations