How to keep network connection persistent during session?

PetaMem has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,

a (Mason-based) webapp connects to a backend server via IO::Socket::INET. Now I would like to make that connection persistent for the session. unfortunately, simply pouring the client object into the session storage doesn't work, as this immediately complains:

The handle is created the usual way - I think.

    $handle = IO::Socket::INET->new(
                                    Proto     => 'tcp',
                                    PeerAddr  => $host,
                                    PeerPort  => $port,
                                    Blocking  => $blocking,
                                );
[download]

But if I want to store it in a session, like this (Dumper output)

   'one-session-key' => bless( {
    ...
               '_handle' => bless( \*Symbol::GEN40, 'IO::Socket::INET'
+ ),
    ...
[download]

I see this error

(in cleanup) Can't store GLOB items at blib/lib/Storable.pm
        (autosplit into blib/lib/auto/Storable/_freeze.al) line 339,
        <GEN40> line 7, at
        /usr/lib64/perl5/vendor_perl/5.8.8/Apache/Session/Serialize/St
+orable.pm
        line 21
[download]

I have read here about why this doesn't work and why some people think it doesn't make sense to try to achieve that persistence. Well - for me the reason is, that when the traffic gets high, I would prefer for a session not always to:

establish ( & authenticate),
send command, receive answer
terminate the connection to backend

Actually I'd be pretty happy if one could omit the 1st and 3rd point, because they are simply performance killers during a session. Now what I have tried;

Circumventing the "Storable cannot store GLOB" problem by transfering bless( \*Symbol::GEN40, 'IO::Socket::INET' ), to a string and later evaluating that actually DOES work, but as I've been told this doesn't keep up the connection, instead it recreates just some glob. The result being a program that astonishingle DOES still work, but I see connections being established and terminated (probably when the last reference to the glob vanishes).

So what could one do? My next ideas would be:

1. some kind of mini-persistent-proxy on the frontend side that would keep up the persistent connections under some accessible index (and the index number of course easily storable in session data.

2. and/or by somehow tieing these sockets to the filesystem (socket files) and using them as the proxy. No idea HOW to do that, just an Idea, because I know there are socket "files".

What would you suggest?

Bye
PetaMem All Perl: MT, NLP, NLU

Comment on How to keep network connection persistent during session? Select or Download Code

Replies are listed 'Best First'.
Re: How to keep network connection persistent during session? by Joost (Canon) on Oct 11, 2009 at 22:38 UTC
HTTP just doesn't work that way. Aside from that, even if you could serialize a network connection (which you can't) it won't help you do what you want to do. If you want to use HTTP, the best easily achievable goal is to use Keep-alive on the connection (which AFAIK any decent webserver should support / suggest / do already by default, but it may help to test it and/or increase the timeout. This won't help you with the authentication part, since HTTP doesn't guarantee that single connections serve only a single session - it's even possible that in between proxies could "bundle" multiple sessions over a single connection (though I don't know of any proxy that does that). In any case, IME if authentication/reconnection takes such a large part of your system, you're either doing it wrong, or you're not doing anything very interesting. If you're really that pressed for processing power, switch to a protocol that's a little less designed for single request/response actions and more designed for long-term connections. Like STOMP. Of course, that means ditching a pure web based approach... "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re^2: How to keep network connection persistent during session? by PetaMem (Priest) on Oct 12, 2009 at 08:15 UTC
Hello Monks, thank you for your answers. I would like to clarify some things. I'm using newest Apache2, newest mod_perl. I do not need nor want to store the connections on disk. I would simply want that within a session a network connection is opened, and remains open as long as the session is active. Which means the network connections life should extend - of course - the life of the running apache processes that happen during that session. That's the whole problem. The backend is not a DB, but an own server process (which uses Net::Server::Multiplex If it's going to be a "daemon brokering the connections", that's ok, I just thought Apache2/mod_perl2 would have something that persistent themself - shared memory perhaps? Some space all Apache processes are sharing for the lifetime of the Apache process? (I'd be willing to take into account that if the Apache as whole had to be restarted that the network connections would be lost). Thanks Bye PetaMem All Perl: MT, NLP, NLU	[reply]
Re^3: How to keep network connection persistent during session? by afoken (Chancellor) on Oct 13, 2009 at 19:38 UTC
I'm using newest Apache2, newest mod_perl. That does not change even one bit in the HTTP protocol, so it is (nearly) irrelevant. I do not need nor want to store the connections on disk. I would simply want that within a session a network connection is opened, and remains open as long as the session is active. Which means the network connections life should extend - of course - the life of the running apache processes that happen during that session. That's the whole problem. Serialisation does not necessarily mean writing to disk. It just means to generate a string of bytes representing a complex structure. And you can't to that with a network connection, because you literally hold only one end of the connection. Basically, you want to extend the session handling to also manage a network connection. A naive implementation would use a hash, using the session ID as key and the file handle of the socket (network connection) as the key. Whenever you create a new session, you also generate a new socket, and store both in the hash. Whenever the session is closed or expires, you close the socket and remove session ID and socket handle from the hash. A better solution would be to separate connection and session also in the backend. If you use some kind of remote procedure call API, create a new session in the backend and get a handle (a simple, random number) back. Use that handle for all interactions during the session, and destroy the handle when the session is terminated or expired. Connect to and disconnect from the backend as needed. The backend is not a DB, but an own server process As most DBs are separate server processes, this is irrelevant, too. In fact, it does not matter if you connect to a DB server, an FTP server, or a coffee machine server. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply]
Re: How to keep network connection persistent during session? by moritz (Cardinal) on Oct 11, 2009 at 21:02 UTC
You simply can't serialize a network connection, and store it on disk. It's something which a program has to negotiate with the operating system, and something that each program has to do anew. If you use something like mod_perl or FastCGI, you can have one process that handles multiple HTTP requests. In that case you can have one socket per process, not just per request. Something like that is probably your best bet. Perl 6 - links to (nearly) everything that is Perl 6.	[reply]
Re: How to keep network connection persistent during session? by sflitman (Hermit) on Oct 11, 2009 at 22:17 UTC
I've had similar issues with any webapp system. Rather than serializing the connection, which just freezes the 'name' of the connection and doesn't really keep it open, the right way to handle this is to use a daemon which runs continuously on the server and brokers the requests to the database, and you're right, uses a FIFO (Unix socket) to talk with your Mason app instances. The daemon can be written in Perl, there are many examples. This way, you can multiplex requests and cache database connections. That said, you might be better off using Apache 2.x, mod_perl, and Apache::DBI which caches connections for you. HTH, SSF	[reply]