4

I'm having trouble investigating an issue with many sleeping MySQL connections.

Once every one or two days I notice that all (151) MySQL connections are taken, and all of them seem to be sleeping.

I investigated this, and one of the most reasonable explanations is that the PHP script was just killed, leaving a MySQL connection behind. We log visits at the beginning of the request, and update that log when the request finishes, so we can tell that indeed some requests do start, but don't finish, which indicates that the script was indeed killed somehow.

Now, the worrying thing is, that this only happens for 1 specific user, and only on 1 specific page. The page works for everyone else, and when I log in as this user on the Production environment, and perform the exact same action, everything works fine.

Now, I have two questions:

  1. I'd like to find out why the PHP script is killed. Could this possibly have anything to do with the client? Can a client do 'something' to end the request and kill the php script? If so, why don't I see any evidence of that in the Apache logs? Or maybe I don't know what to look for? How do I find out if the script was indeed killed or what caused it?

  2. how do I prevent this? Can I somehow set a limit the to number of mysql connections per PHP session? Or can I somehow detect long-running and sleeping mysql connections and kill them? It isn't an option for me to set the connection-timeout to a shorter time, because there are processes which run considerably longer, and the 151 connetions are used up in less than 2 minutes. Also increasing the number of connections is no solution. So, basically.. how do I kill processes which are sleeping for more than say 1 minute?

Best solution would be that I find out why the request of 1 user can eat up all the database connections and basically bring down the whole application. And how to prevent this.

Any help greatly appreciated.

6
  • What exactly is your setup? PHP as apache module or FCGI? Are there information in any of the affected logs? May 23, 2014 at 9:30
  • How many concurrent connections is your webserver configured to handle? Maybe there are more web processes allowed, which each can connect to MySQL, than max_connections of MySQL can handle? This could result in exceeding the limits, without arbitrary killing any processes. May 23, 2014 at 9:43
  • The described behaviour is not normal with default configuration, I suggest not to change configuration, instead try to find the root source of the problem.
    – Daniel W.
    May 23, 2014 at 9:46
  • 2
    @DanFromGermany If the webserver can create more processes than MySQL can handle, this would be pretty normal behavior. Since the killing of PHP processes is still just a guess of the questioner... but I admit taking a closer look makes sense. May 23, 2014 at 9:56
  • I use PHP as an Apache module. It seems that Apache can handle more connections than mysql, so that's indeed the reason that in the end the database refuses any more and new connections start to fail. So I'll update those settings, causing new requests to be queued. However, I still don't know why the mysql connections go to sleep, and why they are not terminated after the Apache session ends...
    – Myrddin81
    May 26, 2014 at 13:13

1 Answer 1

3

You can decrease wait_timeout variable of the MySQL server. This specifies the amount of seconds MySQL waits for anything on a non-interactive connection, before it aborts the connection. The default value is 28800 seconds, which seems quite high. You can set this dynamically by executing SET GLOBAL wait_timeout = X; once.

You can still increase it for cronjobs again. Just execute the query SET SESSION wait_timeout = 28800; at the beginning of the cronjob. This only affects the current connection.

Please note that this might cause problems too, if you set this too low. Although I do not see that much problems. Most scripts should finish in less than a second. Setting wait_timeout=5 should therefore cause no harm…

1
  • Thanks. I'll try this. It's not perfect, but it might just solve my problem for now. Still need to find out why it's happening, because the user is submitting the same requests a little faster, I'm stuck with the same problem again. But thanks for pointing this out.
    – Myrddin81
    May 23, 2014 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.