aka1 has asked for the wisdom of the Perl Monks concerning the following question:

This isn't a technical question so much as a request for opinions. Yesterday I had reason to have perl parse the output of a long running process that only produced occasional output. It seemed to me the best way to go about this was to read from a pipe in a non-blocking fashion, but how to do this (or even if it is possible) without actually setting up the "reader" as a separate process communicating via IPC (e.g. sockets) is non-obvious.

I ended up redesigning my process to take advantage of blocking, but the question still remains an intriguing one. Does anyone out there in the pews have any ideas on simple non-blocking pipes or command execution in general? I am very used to --full-- IPC but I'm looking for something lighter. I am looking for something more akin to how you can open a "normal" filehandle with a O_NONBLOCK flag or a more straightforward way to apply the select() function to pipes.

Thoughts?

Replies are listed 'Best First'.
Re: Non-blocking Pipes?
by Celada (Monk) on Dec 30, 2005 at 05:43 UTC

    Sure, setting pipes to nonblocking mode is not a problem. The fcntl paragraph in the perlfunc manpage tells you how to do it. For checking the filehandle for available input or output, consider using the IO::Select module instead of select directly. Who knows, perhaps the underlying implementation is in terms of the poll system call if it is available and you will not have to worry about the limitations of select (and if it isn't, then it could be!).

[reply]
[d/l]
[select]
Re: Non-blocking Pipes?
by sgifford (Prior) on Dec 30, 2005 at 06:39 UTC
    The details of this are likely to depend on your platform. For example, Windows only supports select on sockets, and fcntl's powers can vary between systems. What OS are you running on?

    Also, anything you can do with pipe can also be done with socketpair, which should support select (and IO::Select) on most platforms. It's worth a try.

[reply]
[d/l]
[select]
Re: Non-blocking Pipes?
by gloryhack (Deacon) on Dec 30, 2005 at 05:36 UTC
    Hmmm... is shared memory out of the question? If not, you might give a look at IPC::Cache.
[reply]
      That's exactly the sorting of thing I was looking for. I've used shared memory before and have found it much easier than socket based IPC. I can't believe I didn't think of that myself. It's a great suggestion.
[reply]
Re: Non-blocking Pipes?
by pileofrogs (Priest) on Dec 30, 2005 at 17:12 UTC

    This is probably more of a question than a response:

    The way I'd do it would be like this:

    use strict;
use warnings;
use FileHandle;

my $handle = FileHandle->new("/path/to/process |");
$handle->autoflush(1);

while(my $line = <$handle>) {
        print $line;
        # do something useful with $line...
}

    [download]

    But I assume that doesn't work for some reason. Can anyone tell me why?

[reply]
[d/l]
Re: Non-blocking Pipes?
by Mr_Person (Hermit) on Dec 30, 2005 at 17:52 UTC
    IPC::Run is great for working with the output of processes in a non-blocking way. If you want something fancier, you can use POE::Component::Child, which will let you use POE to work with child processes and process the output with event handlers.
[reply]
      I've never actually heard of IPC::Run, but it does indeed look like a possible way to go. I am aware that what I want to do is much more trivial in POE. I need to actually get around to learning and using it. :)
[reply]

Back to Seekers of Perl Wisdom