xafwodahs has asked for the wisdom of the Perl Monks concerning the following question:

I want to capture STDOUT and STDERR of another process just like I see it when it gets dumped to the terminal (same order).

I wrote a couple test scripts:

outerr.pl
print STDOUT "out 1\n";
print STDOUT "out 2\n";
print STDOUT "out 3\n";

print STDERR "err 4\n";
print STDERR "err 5\n";
print STDERR "err 6\n";

print STDOUT "out 7\n";
print STDOUT "out 8\n";
print STDOUT "out 9\n";

print STDERR "err A\n";
print STDERR "err B\n";
print STDERR "err C\n";

[download]
when run, this script produces the obvious results (all numbers in order).

So I try to capture it all with open3, this way:

use IPC::Open3;
use Symbol;

$OUTERR = gensym();
$IN = gensym();

print STDOUT "running...\n";
$pid = open3($IN, $OUTERR, $OUTERR, 'outerr.pl') or die $!;
while ( wait != $pid ) {};
print STDOUT "done...\n";

while (<$OUTERR>) {
   print STDOUT "got: $_";
}

close OUTERR;

[download]

However, this does not preserve the order. Strangely, in windows, I get all 6 STDOUT's, then all 6 STDERR's - and on unix, I get all 6 STDERR's, then all 6 STDOUT's.

What gives?

Replies are listed 'Best First'.
Re: IPC::Open3 confusion
by BazB (Priest) on Jan 31, 2003 at 21:20 UTC

    Have a look at IPC::Open3 woes - particularly abstracts(++!) solution using IO::Select to loop through the various filehandles.

    The basic idea of the code is as follows: 

    #!/usr/bin/perl

use strict;
use warnings;

use IPC::Open3;
use IO::Select;
use Symbol;

my $IN  = gensym();
my $OUT = gensym();
my $ERR = gensym();

print "Running...\n";

my $pid = open3($IN, $OUT, $ERR, 'outerr.pl') or die $!;

my $sel = new IO::Select;
$sel->add($ERR,$OUT);

my $data;

while (my @ready = $sel->can_read) {
  foreach my $fh (@ready) {
    read $fh, $data, 8196;

    die "Error from child: $!\n" if ! defined $data;

    if (length $data == 0){
      # No data read - child must be finished writing.
      $sel->remove($fh);
      next;
    } else {
      # read OK:
      print $data;
    }
  }
}

print "Done.\n";

    [download]
    You should read abstracts' node for a fuller explaination.

    I'm not sure if, or how, buffering of STDOUT in your test script will affect things.
    A more enlightened monk will fill in the details :-)

    On my box (This is perl, v5.8.0 built for i386-linux-thread-multi and IPC::Open3 version 1.0104), I always get STDOUT messages before STDERR, regardless of how the buffering on STDOUT is set in outerr.pl.

    BTW, you should consider using strict and warnings.

    Hope this helps.

    BazB

    Update: swapped $ERR and $OUT. Doh. pg++.
    Corrected comments about the output order.

    If the information in this post is inaccurate, or just plain wrong, don't just downvote - please post explaining what's wrong.
    That way everyone learns.
[reply]
[d/l]
[select]
Re: IPC::Open3 confusion
by tall_man (Parson) on Jan 31, 2003 at 21:19 UTC
    It has to do with flushing the output buffers, which happens automatically when you write to the console but not to your special handles.

    Try adding this at the top of your outerr.pl: 

    use IO::Handle;
STDOUT->autoflush(1);
STDERR->autoflush(1);

    [download]
    Update:Also, IPC::Run is reputed to be safer to use than IPC::Open3. Here is how it would look: 
    use strict;
use IPC::Run qw(run);

my @com = qw(outerr.pl);
my ($in, $out);
run( \@com, \$in, \$out, \$out);
print $out;

    [download]
    (Sad to say though, neither this version nor the one by BazB seems to give the interleaved output you want, even with autoflush turned on, on my Linux box. All the STDOUTs come before all the STDERRs).
[reply]
[d/l]
[select]
Re: IPC::Open3 confusion
by pg (Canon) on Jan 31, 2003 at 22:13 UTC
    Two errors:
    1. in outerr.pl, autoflush is not set,. Just do this: 
      use IO::Handle;
STDERR->autoflush(1);
STDSTD->autoflush(1);

      [download]
    2. the sequence of parameter to open3 is: out, in, err. You gave the wrong sequence. This is something confusing, have to be careful, as the sequence of the first two parameters to open2 and open3 are reversed.
    I tested it, and it worked.
[reply]
[d/l]
      Unfortunately, I don't have the luxury of changing the 'called' script.
[reply]
Re: IPC::Open3 confusion
by zentara (Cardinal) on Feb 01, 2003 at 19:27 UTC
    I want to capture STDOUT and STDERR of another process just like I see it when it gets dumped to the terminal (same order).

    I'm not an expert on IPC3::Open , but here is a trick from the docs which combines STDERR to STDOUT by setting STDOUT to false. 

    #!/usr/bin/perl
use warnings;
use strict;
use IPC::Open3;

#interface to "bc" calculator 
#my $pid = open3(\*WRITE, \*READ, \*ERROR,"bc"); 
my $pid = open3(\*WRITE, \*READ,0,"bc");
            #if \*ERROR is false, STDERR is sent to STDOUT  
while(1){
print "Enter expression for bc, i.e. 2 + 2\n";
chomp(my $query = <STDIN>);

#send query to bc 
print WRITE "$query\n";

#get the answer from bc 
chomp(my $answer = <READ>);

print "$query = $answer\n";
}

    [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom