dash2 has asked for the wisdom of the Perl Monks concerning the following question:

Some context: I am trying to write a framework for a Perl peer. A slightly out of date version is at Net::Distributed. My previous woes are at HTTP::Daemon and broken pipes.

My problem is that as soon as a network of peers gets a lot of messages passing around, some peer gets a SIGPIPE. Then no further messages get through to that peer, and lots of other peers start getting SIGPIPEs, and soon the whole network is fscked.

I initially thought that I was somehow handling SIGPIPEs wrongly, so that after a SIGPIPE the peer wouldn't recover. But nobody has pointed out any problems with my code in this area. So another possibility seems likely.

My code doesn't fork. It just comes round once in a while to Net::Distributed::Transport->_accept_messages(), which uses a HTTP::Daemon to listen. I set timeouts, and a listen queue, for the Daemon, which is a persistent object.

I think the problem is this. Peer X tries to send a message to peer Y and - for any reason - can't get through. So it times out after write timeout wt. By this time, peer Z is trying to connect to peer X, and has also almost reached the end of it's own wt. Worse still, maybe peer Y wants to talk to peer X - for various reasons this is fairly common in the sort of network I am playing with - and it too times out because peer X is trying to write to it! End result: all my peers lock up trying to talk to each other, and none of them ever get through.

I ought to know whether this is really the problem, but I am not yet sure. It is bloody hard trying to debug 10 simultaneous peers all chatting to each other in realtime over the loopback. So one question is - how do I do realtime debugging on a network peer?

More centrally: if I am right, I need a solution. I need my peer to listen all the time, so that other peers don't normally time out.

I could fork but the problem is that my peer process needs to know what message has been sent to it - it's not like a web server which can just dumbly serve information, while storing state in a database backend. What the peer does depends on what other peers say to it.

I could fork and have a separate listener process which writes to a file, and the main peer reads messages from that file. But this touches the hard disk, which I am loth to do, and just seems fundamentally ugly and wrong.

I could fork and open a pipe to the listener child. But how can I stop the listener blocking while it tries to write to the parent (which is away doing other things)?

Or I could do something brilliant and wondrous. Maybe you have an idea.

dave hj~

Replies are listed 'Best First'.
Re: Listening without forking
by zengargoyle (Deacon) on Feb 23, 2002 at 00:30 UTC

    Check out the Event module. I've had pretty good luck using it for this type of thing.

    # untested, almost verbatum from working code.
use Event;
use IO::Socket;
my @peers;
my $sock = IO::Socket::INET->new(Listen=>5,LocalPort=>8001,Reuse=>1)
  or die "[Fatal] no getum socket!\n";
Event->io(
  fd=>$sock;
  cb=>sub {
    my $e = shift;
    my $h = $e->w->fd;
    my $peer = $h->accept;
    push @peers, $peer;
    $peer->autoflush;
    Event->io(
      # todo:  data => '<peername>' or such to id peers.
      fd=>$peer,
      cb=>sub {
        my $e=shift;
        my $h=$e->w->fd;
        # todo:  my $name=$e->w->data;
        my $line = <$h>;
        foreach (@peers) { $_->print "$line" } # from $name...
        if ($line =~ /^quit/i or eof($h)) {
          $e->w->cancel();  # todo: remove self from @peers
        }
      }
    );
  }
);

Event::loop();

    [download]

    There's much more in the module, io (r/w/e), signal, timer, idle, var (r/w) handlers...

[reply]
[d/l]
Re: Listening without forking
by lestrrat (Deacon) on Feb 22, 2002 at 19:25 UTC

    Does the message handling need to be synchronous? I'm not sure if this is useful in your context, but if it can be used asynchronously,perhaps you could have the parent and the child sharea message queue, and establish a Producer/Consumer relationship

    So the child would be listening to all incoming requests. When the child gets something, it writes the request to the queue. The parent would check the queue for any available messages, and does the processing.

    At least this way you get to detatch the actual processing with the peer interaction.

[reply]
Re: Listening without forking
by Xanatax (Scribe) on Feb 24, 2002 at 00:05 UTC
    I've heard before that the Event module is the way to go... I've not played with it, but I have tried out IO::Select, and it seemed worthy.

    IO::Select runs as an event loop blocking on the select() command until any specified handly (typically any read handle) handles become 'active' by having data available. you can perform non-blocking reads with this quite well, so the only time the server is blocking is when nothing is happening, or when it is actively getting data from a buffer, which is a fast operation.

    needless to say this causes other intereseting problems in coding, like dealing with data when it is more likely than not incomplete from the non-blocking read... but things like timeouts should be much less of a problem.
[reply]

      In the Event Tutorial (maybe POD doc) there was mention of the blocking problem, recommendation was IPC::LDT or similar. Use messages that include the total length at the beginning of the message. 

      read SOCK, $len_raw, 4;
$len = unpack 'N', $len_raw;
read SOCK, $msg, $len;

and

$len = length $msg;
$len_raw = pack 'N', $len;
write SOCK, $len_raw, 4;
write SOCK, $msg, $len;

or

write SOCK, "$len_raw$msg", $len+4;

      [download]

      Something like that should handle the simple cases.

[reply]
[d/l]
Re: Listening without forking
by dash2 (Hermit) on Feb 25, 2002 at 12:47 UTC
    lestrrat : I think this seems like a good solution. Would the message queue be a pipe? If so, can I make sure that the child transport isn't blocked waiting for the parent to read from the pipe? I am a bit unsure about the mechanics of the pipe mechanism.

    zengargoyle et al: I will check out Event. I know of IO::Select - one worry is that I seem to lose the simplicity of HTTP::Daemon and have to get more tangled up in low level stuff. I guess I would just make the Select object persist and then read from it at regular intervals... that could be cool.

    Thanks for your tips.

    dave hj~

[reply]

Back to Seekers of Perl Wisdom