sharing (io::socket) objects with threads

bumby has asked for the wisdom of the Perl Monks concerning the following question:

I have the following basic code structure:

a main program with a thread pool accepting tcp connections all data received from these connections are put on a thread queue which another thread handles

the "other" thread: this thread has got an irc-connection (Net::IRC) with an infinite while-loop which first handles irc-stuff (do_one_loop) and secondly dequeues the previous mentioned queue.

All irc-events are forwarded to another (or actually several other) tcp-connections (the brains). These brains are separate programs which handles the events, generate replies and send these back via tcp to the main-part (which will queue and be dequeued by irc-thread and printed to channel or whatever).

Now, this all works fine. The problem I'm having is that I want my program to, in a good fashion, reconnect to any lost brain connection. So that if any brain is restarted, it will be reconnected again. I've set SIGPIPE to ignore and successfully detect deconnected brain-connections. The brain connections are stored in a hash with connection-numbers as keys (0, 1, 2, ...). Once a connection is detected as lost its key (connection number) is put on a reconnect-queue, which a reconnect thread dequeues and then tries to reconnect to this connection. ($brains->{$key}->{Socket} = IO::Socket::INET->new...)

Now, of course, this doesn't work, since this brains-hash will be a separate copy from the one in the irc-thread. Or at least that's how I've understood it. I've tried sending references to the thread too, but it seems to have the same effect.

So my question basically boils down to this: How can I reconnect IO::Socket::INET connections in a separate thread?

I want this to be done in a separate (detached) thread to avoid the irc-thread to get stuck while trying to connect to brains.

Is this even possible? Or should I just set these connection to nonblocking and call it a day?

Comment on sharing (io::socket) objects with threads

Replies are listed 'Best First'.
Re: sharing (io::socket) objects with threads by TOD (Friar) on May 17, 2007 at 02:25 UTC
that's a thrilling question, but i don't think your approach will work. because in principle it is not possible to share object references between threads. all shared data must be scalars. so maybe you're trying to catch the problem at the wrong end? if i understand you right, your main thread needs to know whether a child thread (one of the brains) closed the connection. the first idea that comes into my mind is: reverse the socket architecture. let the main thread listen and accept connections, not the brains. if this is a trivial proposal to you, there's maybe another thing possible, namely via localizing a hook for $SIG{__DIE__}. when a brain's socket closes the connection, this should be noticed. and the $SIG{__DIE__} handler should then put maybe the brain's identifier on a thread queue, which now effects the main thread to re-establish the connection. i don't know ehether these remarks will be helpful, they just came into my mind. cheers TOD -------------------------------- masses are the opiate for religion.	[reply]
Re^2: sharing (io::socket) objects with threads by BrowserUk (Patriarch) on May 17, 2007 at 03:27 UTC
... in principle it is not possible to share object references between threads ... But, IO::Socket::INET objects are not objects in the traditional perl sense in as much as they are actually globs. And, in common with traditional filehandles which are also globs, it is possible to derive a fileno from them, which is just a number (scalar) and can be passed between threads. Once you have the fileno in the thread, it is possible to dup a new glob from that fileno and so use that to access the socket. However, thread handles are true objects. So, whilst if you have the cpan version* of threads*, signalling between threads is possible, for your scheme to work, the child thread that is going to signal the main thread would require a thread handle in order to call the kill method upon it. How are you going to pass the thread handle (object) to the child thread(s) inorder to do this? It is possible through another round-about mechanism, but you surely haven't identified it. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^2: sharing (io::socket) objects with threads by chrism01 (Friar) on May 17, 2007 at 05:53 UTC
FYI, you can thread-globally share complex structures eg HoHoH ... but you can only send scalars down a Thread::Queue. Although I think I once read that that (latter) was being worked on ... Cheers Chris	[reply]
Re^2: sharing (io::socket) objects with threads by bumby (Beadle) on May 17, 2007 at 11:47 UTC
Letting the brains connect to the irc-proxy-thing instead of the reverse is brilliant! I've most have overseen this due to lack of sleep or something :) A simple yet brilliant solution! Thanks a lot! And thank you all others for your input!	[reply]
Re: sharing (io::socket) objects with threads by BrowserUk (Patriarch) on May 17, 2007 at 01:30 UTC
It may be possible to do what you want to do, though definitely not the way you are currently trying to do it. However, you've described a very elaborate and involved program, far too elaborate to be able to visualise from a brief textual description. And far to complex to expect anyone trying to help you to re-create it from scratch from that description. Upshot: If you want to get any help with this, you are going to have to post the code you have so far in it's entirety. Any thing less is expecting way too much of potential helpers. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re: sharing (io::socket) objects with threads by chrism01 (Friar) on May 17, 2007 at 05:47 UTC
Sounds like you've got 1 output/IRC thr which talks to multiple external progs (brain progs). 1 option is to use a tcp connect with timeout option eg: `if( !$send_socket ) { # Create socket $socket_errmsg = ""; $send_socket = IO::Socket::INET->new( PeerAddr => $server_ip, PeerPort => $server_port, Proto => "tcp", Timeout => $cfg::params{'CONNECT_TIMEO +UT'} ) or $socket_errmsg = "Couldn't create socket to ". "$server_ip port $server_port: + $@"; }` [download] Another is to have 1 IRC thr per brain prog (if you haven't got too many) and do the same. Either try to re-connect from the thr that has a dead cxn, or get it to mark a thr shared hash (%brains) that the cxn is dead, then terminate naturally and get the controlling thr to initiate a new thr to connect to the brain prog. The latter might be pref, as you'll prob have to reset any thr private vars anyway. Cheers Chris	[reply] [d/l]
Re: sharing (io::socket) objects with threads by zentara (Cardinal) on May 17, 2007 at 12:15 UTC
See FileHandles and threads and Passing sockets between segregated threads for examples of BrowserUk's suggestion for passing fileno's between threads. It can be done. It's probably best to open the Socket in the thread, then pass back the fileno of the socket thru a shared variable to the main thread. It's tricky, and probably is easier with an event-loop system in the main thread (like Tk, Gtk2, or POE). Otherwise you will need a complex while loop using IO::Select to watch all the filehandles. I'm not really a human, but I play one on earth. Cogito ergo sum a bum	[reply]