in reply to TCP server: How to reject connections when busy?

So, why didn't you immediately type "man listen" at the shell prompt on your operating system of choice?

When I do that on a similar system, I immediately noticed:

If a connection request arrives when the queue is full, the client may receive an error with an indication of ECONNREFUSED or, if the underlying protocol supports retransmission, the request may be ignored so that a later reattempt at connection succeeds.

Seems a quite stupid feature to allow one to reduce how much one wants to queue up connections in a way that is designed to cause the client side to just queue up connections in a different way. *shrug*

Given that, you'll probably want two streams of execution where one stream does work and the other stream actively rejects connections. There are lots of choices when it comes to having multiple streams of execution in Perl, but each of the choices usually have drawbacks that can become significant. But, for this simple case, you can probably use most of them without problems.

If you can't guarantee one stream has "priority" so that the other stream only receives connections when the first is busy, then you'll probably want a non-blocking semaphore / lock / critical section so each stream can know when it should just reject.

For this case, I'd probably just use threads.pm with a shared variable (despite usually finding myself in situations where Perl "threads" aren't the best solution). Note that you'll need to explicitly lock the shared variable around your 'test and set' code. See also: threads::shared.

- tye        

  • Comment on Re: TCP server: How to reject connections when busy? (some FM TR)

Replies are listed 'Best First'.
Re^2: TCP server: How to reject connections when busy? (some FM TR)
by squirrel (Novice) on Nov 29, 2011 at 21:44 UTC

    The purpose is to return control to the client if the server is busy, so that it can determine what to do.

    The server code referenced previously is a stripped down case. The actual server is using a pre-fork model. Open socket, fork multiple child processes each calling accept with the OS figuring out which child gets to do the accept. Would you use semaphores?

      Whatever proves to be a good measure of "server is busy". If requests are relatively constant in resource cost, then you might have a hard ceiling of "N max simultaneous requests" and a semaphore would be a good fit (and you'd need to pre-fork upto N+1 children, of course).

      But you might have a different "busy" metric you want to use like "system load average" or "idle CPU percentage" or "virtual memory in use" or whatever.

      But, even if you do implement some "busy" metric other than "N simultaneous requests", you'll probably still have some "max N children" configuration and so a N-1 semaphore is probably still a good idea.

      - tye        

      If it's worth for the client to reconnect to another box if the server is busy, I suppose the jobs take relatively long to process, otherwise the client could expect a server process to become free soonish when it enters the listen queue.

      Are you sure you want preforked servers for that? Because the preforking is usually done to avoid the fork() overhead if it takes a significant fraction of the whole request processing time.

      Otherwise you could have one master process that accepts all connections and can optionally have a little dialog with the client. If it can down a nonblocking semaphore, it just forks and lets the child do the rest, otherwise it tells the client to check another box (you could even have the servers inform each other of their current load so it could give a hint to the client) and hangs up.

      I don't think that using semaphores is a particularly good idea. There are some situations where these things can get left in memory when crashes happen and part of a system resource can just without much adieu ... just go away.

      See my other post with "option 2". If you want to share a bit of "go no/go information" between processes, consider using the file lock mechanism. This is a common way to do this. The "busy/not busy decider" manages a write lock.. more specifically an exclusive lock to a zero length file. Other processes test this file to see: "if I wanted to get a write lock, could I do it?". If the answer is "no" then close the active connection.

      The file lock control table is a memory resident thing and checking this is fast but you have to open the file first, however this is usually fast compared with the network delays just to get to your box to begin with. Have the "decider" function be the only one who actually locks this "I_am_too_busy" file.

      If the "decider" process crashes for some reason, its lock is released - no clean-up required. A semaphore can potentially have problems.

      There are other ways to share information between processes on *nix systems. I would think about the easiest ways and only get more complicated when needed. Stay with the *nix forking server model if you can. I would also consider the post by mbethke.

      I don't see any specs on how fast this has to be, nor benchmarks that show why a particular implementation is too slow. The fastest implementation would be a "super server" which is the most complicated model because it involved both select based and fork based code. I would write that in C if this level of complexity and performance is needed.

      Anyway, this thread started with a fairly simple question and it appears that things are getting more and more complicated.