ccarden has asked for the wisdom of the Perl Monks concerning the following question:

Hi, folks,

I don't quite seem to understand how redirection works. I know that 2>&1 feeds stderr into stdout. I think that adding a pipe after that (2>&1|) pipes the combined output back to the file handle (yes/no?). But how do I test failure of the external command? Seems to me that one would not want to combine stderr with stdout in that case, que no?

Here is an example of what I'm currently doing: 
my $cmd = 'someProcess.pl -parm val';
open (CMD, "($cmd  | sed 's/^/STDOUT:/') 2>&1|");
while (<CMD>;) {
    if (!s/^STDOUT://) {
        exitSub(2,"  Process failed: $_");
    } else {
        # Split the output of the process into ret
        chomp;
        @ret = split;
    }
}
close (CMD);

[download]
I'm not convinced that this is the best way to do things. For one thing, I'm don't really dig using sed or any other system command, if I don't have to. For another, I've seen examples of code with die if the open fails: 
open (CMD, "($cmd  | sed 's/^/STDOUT:/') 2>&1|") or die $!;

[download]

Currently, I find that if my external process fails or gives warnings, the calling code (first example) lets it fall through. A recent example was when I added use warn to the external script and it started spewing out some warnings. This caused the calling script to fail because of unexpected text in the resulting capture of stdout. If I recall correctly, warnings go to stdout, not stderr, right? How would you handle warnings, then?

If someone could help me understand what is going on, I would appreciate it. I'd rather learn how to fish than be given one.

Thanks.

Replies are listed 'Best First'.
Re: Error trapping while capturing stdout
by Random_Walk (Prior) on Sep 14, 2004 at 14:37 UTC

    under unix shell 2>&1 redirects STDERR to STDOUT. If you then add a pipe you do indeed get the whole lot going down the pipe (so if this pipe then connected to a perl file handle you would be reading STDERR and STDOUT mixed together in Perl). Perl sends it's warnings to STDERR as this little test shows 

    # A simple perl script to generate a warning
: perl -we 'my $t; my $t'
"my" variable $t masks earlier declaration in same scope at -e line 1.
# the same with STDOUT redirected to null
: perl -we 'my $t; my $t' >/dev/null
"my" variable $t masks earlier declaration in same scope at -e line 1.
# same with STDERR to null, aaah the sound of silence
: perl -we 'my $t; my $t' 2>/dev/null
:

    [download]
    What may be enough for you is to check the exit code of the script you call (hopefuly it exits with 0 when it runs OK and non-zero values for failures 
    my $cmd="ls /no/way"; 
my @data=`$cmd`; 
if($?) {
    print "$cmd failed with exit code $?\n";
    exit 1
}
foreach (@data) {
    chomp;
    my @ret=split;
    # do something with @ret here ?
}

    [download]

    This is running the external script to completion first then checking it did not give an error code before going on to process the results.

    Cheers,
    R.

[reply]
[d/l]
[select]
      Thanks for confirming my understanding of 2>&1|.

      Now you got me thinking, with your second point. The called script does, indeed exit with a code. How would perl -w affect that? If perl gives warnings, it doesn't change the script's exit code. So I could still get an exit code of 0, but with warnings from the interpreter.

      Another unwanted behavior that wasn't trapped was perl exiting with a "bad interpreter" message. I tracked it down to hidden carriage returns (thanks to vi's [dos] message), but the calling script should have caught that. I'll test your suggestion for trapping.

      Thanks.

[reply]
[d/l]
[select]

        You would still get the warnings if you redeirected STDERR to STDOUT but if you trust the called script to exit non-zero when it fails then you no longer need to examine STDERR. You can throw it away with 2>&/dev/null and just test the exit value from the script.

        Fixing the warnings may also help ;-)

        The bad interpreter error would also return a non-zero value to the calling script and get caught in the $? test.

        Cheers,
        R.

[reply]
[d/l]
Re: Error trapping while capturing stdout
by ikegami (Patriarch) on Sep 14, 2004 at 14:23 UTC
    Doesn't that only redirect the STDERR of sed, not of $cmd? In any case, you're better off using IPC::Open3 (read perlipc on how to manage children), do your transformation in perl, and merge your streams (check out select(RBITS,WBITS,EBITS,TIMEOUT) or IO::Select) in perl.
[reply]
      I will look at IPC::Open3 based on your suggestion, but I am now very leary of introducing modules into an solution if they don't come with the standard Perl load. It adds a level of complexity to support. I'm willing to handle that extra complexity, if the module delivers above and beyond expectations. Otherwise, it may be better to stick with core Perl solutions.

      Thanks for your input.

[reply]
        I think it does come with perl; it came with mine, and it's not in site/. It's really just a wrapper for fork()+exec(), so no biggie if you don't use it. Check out the perlipc doc/man page for an example using fork()+exec().
[reply]
Re: Error trapping while capturing stdout
by coreolyn (Parson) on Sep 14, 2004 at 16:17 UTC

    Here's the umpteenth time I'm redirecting to this 0 rep Node.

    For me it has solved my STDERR & STDOUT issues for a couple of years now in a production environment on win32, solaris, and linux platforms. By running this in a module it allows me to error check all of my system level calls back in my main code no matter what I call for any given platform, as well as making passing information to my logs easier.

    I never use system or backticks any more.

    coreolyn
[reply]
      Thanks for the link! It was very helpful.
[reply]

Back to Seekers of Perl Wisdom