in reply to Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file

This is covered in Recipe 16.9 of the Perl cookbook. Use IPC::Open3.
  • Comment on Re: Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file

Replies are listed 'Best First'.
Re^2: Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file
by OfficeLinebacker (Chaplain) on Apr 18, 2006 at 20:40 UTC
    Kloogy solution:
    use IPC::Open3; use IO::Select; $cmd="find /fst/home/ -name '*ooo*'"; $pid = open3(*CMD_IN, *CMD_OUT, *CMD_ERR, $cmd); close(CMD_IN); $SIG{CHLD} = sub { print "REAPER: status $? on $pid\n" if waitpid($pid, 0) > 0 }; $selector = IO::Select->new(); $selector->add(*CMD_ERR, *CMD_OUT); while (@ready = $selector->can_read) { foreach $fh (@ready) { $line = scalar <$fh>; if (fileno($fh) == fileno(CMD_ERR)) {print "STDERR: ", $line} else {print "STDOUT: ", $line} ($line) || $selector->remove($fh); } } close(CMD_OUT); close(CMD_ERR);
    • Annoyance: upon completion there are STDOUT: STDERR: in front of the prompt (easy to clean out in the next step)
    • Observation of strange behavior:
    • The messages only come out in the "right" order the first time the program is run in a shell. Subsequent calls mess up the order. Thus I have to exit and reenter the shell each time I debug. Weird.
Re^2: Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file
by Anonymous Monk on Apr 18, 2006 at 17:54 UTC
    OK I went ahead and created an account. Thanks for all the help so far, guys.

    So when I use open3, I create a program (sniped straight from the recipe only with my find command):

    use IPC::Open3; use IO::Select; $cmd="find /fst/home/ -name '*ooo*'"; system("$cmd"); print "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@@@@@@@@@@@@@@@@@@@@@@@@@@@\n"; $pid = open3(*CMD_IN, *CMD_OUT, *CMD_ERR, $cmd); $SIG{CHLD} = sub { print "REAPER: status $? on $pid\n" if waitpid($pid, 0) > 0 }; #print CMD_IN "This line has a vt33 lurking in it\n"; close(CMD_IN); $selector = IO::Select->new(); $selector->add(*CMD_ERR, *CMD_OUT); while (@ready = $selector->can_read) { foreach $fh (@ready) { if (fileno($fh) == fileno(CMD_ERR)) {print "STDERR: ", scalar +<CMD_ERR>} else {print "STDOUT: ", scalar +<CMD_OUT>} $selector->remove($fh) if eof($fh); } } close(CMD_OUT); close(CMD_ERR);

    In this command, there are several dozen lines of output, and two of them go to STDOUT; the rest go to STDERR. The output looks something like

    find: xxxxxxxx .. /path/to/match1 find: xxxxxxxx .. /path/to/match2 find: xxxxxxxx

    however, running the program above, it simply hangs after the second line of output to STDOUT (the rest of STDERR is "left hanging?")

    Edited by planetscape - added code tags

      • Please the <code> tags around your code. (It's custom to PerlMonks.) It makes it (more) readable, preserves whitespace, handles HTML escaping, wraps lines if needed, and provides a mechanism to view the code unwrapped.

      • One problem I see is that you use <FH> instead of sysread. See the warning at the bottom of the documentation for select.

      • Another problem I see if that it only prints "STDOUT: " or "STDERR: " for every chunk, not necessarily every line. You need to split on newlines what you get.

      • I never figured out the use of some methods in IO::Select. What's the use of can_read if you can't check for errors? The proper usage seems to be:

        use IO::Select; $selector = IO::Select->new(); $selector->add(*CMD_ERR, *CMD_OUT); while ($selector->count()) { my ($readable, undef, $error) = IO::Select::select($selector, undef, $selector); $selector->remove($_) foreach @$error; foreach my $fh (@$readable) { ... } )
      • I don't know what you mean by "left hanging".

        OK, sorry about the code tags thing. I just tried it for the output of the strace (I can't edit the posts I made as Anynymous). So the code using
        print "STDXXX: " , scalar <FH>
        is straight from the recipe in the Perl Cookbook! Anyway, how do I use sysread instead of scalar <FH> ? As far as left hanging, I mean the program simply stops executing and the rest of the STDERR output is sort of left hanging in the ether, I guess, never to be outputted.
      Ok so I ran an strace on the program and the last lines are

      <p> read(5, "/fst/home/m1jas05/.openoffice/sh"..., 4096) = 45 write(1, "STDOUT: /fst/home/m1jas05/.openo"..., 53STDOUT: /fst/home/m1 +jas05/.openoffice/share/dict/ooo ) = 53 read(5, <p>

      so it looks like some kind of deadlock, but the child is not reading anything, only the parent is.

      Man I am confused.