Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,
I am running a system command with backticks, to capture output. However some stderr results were being excluded. So I added a "2>&1" at the end of the system command. this gives me all the output the same as running in a shell prompt, except that the stderr stuff is at the top, and stdout is at the bottom of the returned data. I would like to capture this data as it occurs, the same as it would appear as if I ran the command in my shell.
Thanks,
Jonathan

Replies are listed 'Best First'.
Re: STDERR and STDOUT
by liverpole (Monsignor) on Nov 18, 2005 at 14:06 UTC
    The problem you're having is with buffering.  There is no guarantee that you will get STDERR results and STDOUT results in the same order each time; in fact, it's quite likely to change from run to run.  Sometimes you might get all of your STDOUT at the bottom, and other times it might be interleaved.

    If you really need to capture both STDERR and STDOUT, and especially if you care about the ordering, using "backticks" (` `) may not be the best way to go.  Why don't you give an example of what you want to achieve, and especially some of the code you've tried, and maybe we can help you brainstorm other solutions.


    @ARGV=split//,"/:L"; map{print substr crypt($_,ord pop),2,3}qw"PerlyouC READPIPE provides"
Re: STDERR and STDOUT
by Perl Mouse (Chaplain) on Nov 18, 2005 at 14:06 UTC
    That won't be easy to do. You're suffering from buffering, but not in your program, but in the program you are calling. Typically, STDOUT is buffered, while STDERR isn't. Only when STDOUT is redirected to a terminal, it's flushed when a newline is emitted. But you don't call it from a terminal - you're capturing the output, so the output is buffered, and appears later than the unbuffered STDERR.

    If you can tell the called program to run output unbuffered, than that's your easiest option.

    Perl --((8:>*

      If asking it nicely is not possible, you can still run the program on a pseudo-TTY to make it behave as it would on the console. It’s not much fun for such a simple task though, even if IO::Pty will do the dirty work for you.

      Makeshifts last the longest.

Re: STDERR and STDOUT
by jeffa (Bishop) on Nov 18, 2005 at 15:33 UTC

    I would use IPC::Open3 for this. You can open two filehandles simultaneously, one for STDOUT and one for STDERR and collect their respective outputs into two separate scalars:

    use strict; use warnings; use IPC::Open3; my ($msg_out,$err_out); eval { open3(undef, *OUT, *ERR, @ARGV) }; if ($@) { $msg_out = ''; $err_out = "invalid command '$ARGV[0]'"; } else { $msg_out = do {local $/;<OUT>}; $err_out = do {local $/;<ERR>}; } print "STDOUT: $msg_out\n"; print "STDERR: $err_out\n";
    Just run this script and pass the command (and options) you want it to run for you, for example:
    ./foo.pl ls -la
    
    ./foo.pl some_command_that_does_not_exist
    
    ./foo.pl perl -le 'die "goodbye cruel world"'
    
    etc. IPC::Open3 does a lot more than just this, be sure and read the docs as you might need to wait for and reap the child process yourself. YMMV. :)

    UPDATE: Good point, Aristotle. This code indeed does not record STDOUT and STDERR as they occur, but is that really a reasonable request? Have you ever had need for such? I can't say that i have ... :-/

    jeffa

    L-LL-L--L-LL-L--L-LL-L--
    -R--R-RR-R--R-RR-R--R-RR
    B--B--B--B--B--B--B--B--
    H---H---H---H---H---H---
    (the triplet paradiddle with high-hat)
    

      I've done this, passing the same (undefined) scalar for the STDERR and STDOUT of the child process. I don't know if it works everywhere (and I haven't tried it on a non-Unix machine), but it seems okay to me so far where I've tried it.

      Note that this doesn’t give him his STDOUT and STDERR interleaved as they would be on the console; which he seems to want.

      Makeshifts last the longest.

Re: STDERR and STDOUT
by radiantmatrix (Parson) on Nov 18, 2005 at 14:44 UTC

    You might have better luck with using the pipe form of open. Read up on perlopentut.

    open COMMAND, 'my_console_app --option |' or die ("Can't open pipe to +command: $!");

    In that case, you'll only get STDOUT, but read the docs above to learn about getting what it is you want.

    <-radiant.matrix->
    A collection of thoughts and links from the minds of geeks
    The Code that can be seen is not the true Code
    "In any sufficiently large group of people, most are idiots" - Kaa's Law
Re: STDERR and STDOUT
by OfficeLinebacker (Chaplain) on May 20, 2006 at 06:04 UTC
    Greetings, esteemed monks!

    As long as you don't care about distinguishing it (like printing an STDERR: in front of it or something), just do as chromatic says, and use open3() and give it the same FH for both stdout and stderr.

    Now I, on the other hand, have been fooling around trying to find a solution where you can do something different to the STDERR output (like surround it with HTML tags), but print both types of output to the same file, and keep all the statements in the exact order you would get them if you were running them interactively.

    The closest you will get for that is IPC::Run, which will work fine as long as the statements aren't coming rapid-fire and alternating a lot between STDOUT and STDERR. _________________________________________________________________________________
    Without me, it's just aweso