Sleeping mysql connections after finish and disconnect

drusher has asked for the wisdom of the Perl Monks concerning the following question:

I am fastidous about finish statements after each query and have automated the db disconnects at the end of my perl script. I have no persistent connections. All code is perl with no client side code.

Yet mysql runs very slowly with a lot of sleeping connections.

I am seeing a lot of sleeping connections in show processlist that run for extremely long time after perl exits. Even after restarting mysql it loads up pretty quickly

I am running mysql 5.1.73 on redhat linux VM. All except one of my tables are InnoDB

the code runs serially with no child processes and no more than one query being executed at a time.

wait_timeout is set to default 28800 which is obviously ridiculous. I would rather not solve the problem by killing connections since I have a couple of nighttime cron functions that take a long time to run.

It seems to me that if the connections are not closing as they should when I finish/disconnect there might be something else going on. It there by any chance a bug in 5.1.73 where mysql disconnects are not processed?

Any ideas?

Comment on Sleeping mysql connections after finish and disconnect

Replies are listed 'Best First'.
Re: Sleeping mysql connections after finish and disconnect by fishmonger (Chaplain) on Sep 15, 2015 at 20:32 UTC
What is the interactive_timeout set to? Have you tried lowering it? I'd lower both of the timeouts to a more reasonable level. Do you have the slow query log enabled? Are most of the sleeping connections from the same host?	[reply]
Re^2: Sleeping mysql connections after finish and disconnect by drusher (Initiate) on Sep 18, 2015 at 15:59 UTC
We have slow queries log enabled and Icinga monitoring the server. The precipitating issue is that we have a table that has too much data for InnoDB so it is MyISAM, which is real slow in a multiuser environment due to table locking Peak server usage is fairly consistent from day to day, but the problem comes and goes without an explanation. For now I moved some records to history so we have about 29k records instead of 38k records in the MyISAM table. We have a couple of hosts making calls, but the majority of the hanging queries come from one application that makes queries to a big table. This does not explain why we have queries against this one big MyISAM table that are apparently HUNG and run for hours. This does not make sense when all queries are finished and dbconnects always done. I think that when perl signals the disconnect, MySQL / ISAM does not always get the signal to close the connection. It looks like the connection is sitting there doing nothing while tying up resources. There was a PHP bug similar to this -- so I am wondering if Perl has the same bug?? We are playing with wait_timeout in development. So far 600 works without killing crons that take a while to run. Obviously this is the only possible fallback fix. Now that InnoDB is launched and working, we are hoping that the MySQL team will burp the InnoDB max data size up to 384k (or the same as MyISAM whatever that is). InnoDB should work for any existing MyISAM table. Unless there is some reason above my pay scale, it does not make sense that row locking would require a smaller data size. Table locking is a method (in my book) and is not a particularly wise method in the majority of systems which are mult-user.	[reply]
Re^3: Sleeping mysql connections after finish and disconnect by nikosv (Deacon) on Sep 18, 2015 at 20:57 UTC
Could be one or more of related bottlenecks: Do you have optimized statistics for the table? are you using transactions?do you exclusively lock the whole table or do do use page level locking? If you lock the whole table exclusively for each transaction you do have a big issue. There is also possibility that transactions within a session are getting hung and being zombied but not the session itself ISAM tables are notorious for overflow pages,better try a btree structure Are your queries using the indexes? are you using functions in the where clause? what is the query execution plan's estimation on Disk I/O resources for the given queries?	[reply]
Re^3: Sleeping mysql connections after finish and disconnect by poj (Abbot) on Sep 18, 2015 at 16:47 UTC
This post suggests sleeping connections are not always a problem. This does not explain why we have queries against this one big MyISAM table that are apparently HUNG and run for hours. Have you looked at the output from EXPLAIN poj	[reply]