IO::Socket doesn't detect lost TCP connections

tjdmlhw has asked for the wisdom of the Perl Monks concerning the following question:

I was recently given the task of writing a perl script that would accept transactions from a MQSeries Queue and send them to a TCP/IP Socket. The socket had to remain connected for multiple sends through out the day because if the receiving system loses connectivity, user intervention is required to restart it.

The script was written and tested (see cut down version below), but I encountered a problem that I am having trouble getting around. If the receiving system loses connectivity and then is restarted, the perl script doesn't recognize that the connection is no longer there. The script sends the transaction without receiving an error message and goes back to get it's next transaction. The receiving system is waiting for the perl script to re-establish the connection. The script does fail on the second attempt at sending a transaction, but there should be some way of detecting the problem before then.

I have searched Monks, in CPAN, perldoc, several other perl sites, and whatever books I can get my hands on. All to no avail. It probably is a simple command that I am just overlooking because of my limited perl abilities, so I am hoping that I am not asking to much of the monks.

#!/quovadx/qdx5.1/integrator/bin/perl -w
use IO::Socket;
use strict;
$/ = "\x1c\x0d";
my $sockout;
$sockout = new IO::Socket::INET (
               PeerAddr => "eng1tst",
               PeerPort => "9903",
               Proto => 'tcp',
               Timeout => 10
              );
die "Could not create socket: $!\n" unless $sockout;
while (1) {
       my $msgs = `/hci/qc -IMILTP.CLHADT.RADTG`;
       my @msgs = split(/\n/,$msgs);
       my $msg;
       foreach $msg (@msgs) {
               print $sockout "$msg";
               my $rply = <$sockout>;
               print STDOUT "$rply\n";
       }
       sleep 10;
}
close($sockout);
[download]

20040903 Janitored by Corion: Replaced TD/TR tags by P tags

20040904 Edit by castaway: Changed title from 'TCP/IP Question'

Comment on IO::Socket doesn't detect lost TCP connections Download Code

Replies are listed 'Best First'.
Re: IO::Socket doesn't detect lost TCP connections by NetWallah (Canon) on Sep 03, 2004 at 18:31 UTC
The traditional solution to this is to send and recieve "keepalive" packets every few seconds. If these do not show up in time, close, and re-open the connection. Earth first! (We'll rob the other planets later)	[reply]
Re^2: IO::Socket doesn't detect lost TCP connections by tjdmlhw (Acolyte) on Sep 04, 2004 at 02:34 UTC
The receiving system is a vendor package and keepalive heart beats are not part of their system. Once this script is functioning at a production level, variations of it will be used for multiple systems. Out of the ones that I am familiar with, only one uses a heartbeat. It would be prohibitive to pay all of the vendors to modify their systems.	[reply]
Re^3: IO::Socket doesn't detect lost TCP connections by DaveH (Monk) on Sep 04, 2004 at 16:55 UTC
I have implemented similar gateway interfaces in the past, and one thing to bear in mind is that a "keep alive" message does not need to be an explicit thing written into the TCP/IP protocol you are using. Anything which will test whether or not a connection is alive is sufficient. For example, you may be able to send some sort of benign transaction which is valid in the currently defined protocols, but doesn't actually have any effect on the running backend system (i.e. is some sort of read-only query). The would serve the same purpose as an explicitly written keep-alive packet. Obviously, you will need to investigate whether such a query exists and would be suitable for this purpose. The other thing which I would do is to not use the buffered input and output functions for doing production socket work. I prefer using sysread/syswrite for socket reads and writes because you can detect things like dropped connections and end of file conditions more easily. You can also see how much data was read or written to the socket with each operation, so you can detect short read/write conditions. Also, I prefer using IO::Select to see whether my sockets can be read from or written to, and I use this to implement my own timeout mechanism. However, I would recommend seaching on CPAN for TCP, since this threw up lots of interesting higher-level modules which should hide some of the low-level socket guts from your program. Also I would always recommend looking at POE for any socket programming work, since many socket-based programs fall into the event driven category (i.e. "wait for X condition, then do Y"), and POE is one of the best frameworks for achieving event based programming quickly. It is worth having a look at the POE website in addition to the POD documentation on CPAN. You also mentioned that you are using MQSeries for your message queueing. Are you aware that there is an MQSeries Perl module available on CPAN? This may be more suitable than calling out to an external program to retrieve messages. Find below the sort of code I have used in the past for socket operations: use this at your own risk, and bear in mind that this is not working code. You will need to customise it. Provided in the hope that it is useful. Read more... (4 kB) I hope that this helps. Cheers, -- Dave :-) $q=[split+qr,,,q,~swmi,.$,],+s.$.Em~w^,,.,s,.,$&&$$q[pos],eg,print	[reply] [d/l]
Re^4: IO::Socket doesn't detect lost TCP connections by tjdmlhw (Acolyte) on Sep 04, 2004 at 21:42 UTC
Re^4: IO::Socket doesn't detect lost TCP connections by Anonymous Monk on Sep 04, 2004 at 17:20 UTC
Re^4: IO::Socket doesn't detect lost TCP connections by tjdmlhw (Acolyte) on Sep 07, 2004 at 19:18 UTC
Re: IO::Socket doesn't detect lost TCP connections by kscaldef (Pilgrim) on Sep 04, 2004 at 02:10 UTC
Well, the first thing I would do is start checking the return value of `print`. If it is false, then you'll want to look at `$!`. There are a number of possibilities here that you may want to handle in different ways.	[reply] [d/l] [select]
Re^2: IO::Socket doesn't detect lost TCP connections by tjdmlhw (Acolyte) on Sep 04, 2004 at 02:42 UTC
This is one of the first things that I tried. The return value is true and the $! returns blank on the first print. Thanks for the suggestion though, keep them coming. A search of Google turned up an article about $SIG(PIPE) as being a way of determining if a connection is lost, but I don't know how to use this. Is anyone familiar with the $SIG(PIPE)? Could it be a solution to the problem?	[reply]
Re^3: IO::Socket doesn't detect lost TCP connections by kscaldef (Pilgrim) on Sep 04, 2004 at 16:23 UTC
Your process will get a SIGPIPE if you attempt to write to a socket that is closed. However, the default action for SIGPIPE is to terminate your process (with no error message!), so that probably isn't what's happening to you. However, that's not to say that it probably wouldn't be a good idea to set up a SIGPIPE handler to prevent this in the future. If you are not familiar with this stuff already, then you might want to get your hands on Stein's book on network programming in Perl, and Stevens on Unix network programming.	[reply]
Re: IO::Socket doesn't detect lost TCP connections by zentara (Archbishop) on Sep 04, 2004 at 15:16 UTC
I'm not an expert, but I've played around with some TCP connections. I was looking at the perldocs for IO::Socket and noticed the $socket->connected method, which tests if the socket is in a connected state, and will return undef if not connected. So maybe you can test for it in your loops. I'm sure if you dig around thry the various methods available on for a $socket you can find a workable test. I'm not really a human, but I play one on earth. flash japh	[reply]
Re^2: IO::Socket doesn't detect lost TCP connections by tjdmlhw (Acolyte) on Sep 07, 2004 at 19:22 UTC
The $socket->connected doesn't work for detecting if a socket has been lost. It's explained in another article - $Socket->connected Not Returning False? Thanks for your reply, I'm still looking so any other ideas are welcome.	[reply]
Re^3: IO::Socket doesn't detect lost TCP connections by zentara (Archbishop) on Sep 08, 2004 at 13:23 UTC
Still looking? Dosn't the can_read or sysread examples in $Socket->connected Not Returning False? work for you? I'm not really a human, but I play one on earth. flash japh	[reply]
Re^4: IO::Socket doesn't detect lost TCP connections by tjdmlhw (Acolyte) on Sep 08, 2004 at 23:59 UTC
Re^5: IO::Socket doesn't detect lost TCP connections by zentara (Archbishop) on Sep 09, 2004 at 13:16 UTC


Pathologically Eclectic Rubbish Lister
	PerlMonks