I also suspect there is an easier way to accomplish my goal. Well, there's no better place to ask!
Background: A colleague with a large code base reached out to me for help. There are places in his code where he executes a series of piped Unix cmds, like "cmd1|cmd2|cmd3". If any cmd in the series fails, he wants to identify it correctly. This can't be done with a regular system call because it only gives you the status of the last cmd in the series. Enter IPC::Run. It worked like a charm until we started having "pipe out" cmds (not sure what the proper term is), like "|cmd1|cmd2|cmd3", where he is opening the FH for the cmd first and passing data to it later. I have to redesign the module that runs IPC::Run, and that's where I'm stuck. BTW, since the task is to implement the feature in his existing (and complicated) code, I want to wrap all the logic into my modules to keep his code change to a minimal
I've narrowed down the problem to this: after IPC::Run accepts the cmds, it runs each as a child process. The finish() method of the harness object (i.e. $ipc_run_h->finish()) calls waitpid() on each child cmd. If it gets -1, then it sets $? to "unknown result, unknown PID". This is what I'm getting, and I figured out after a while it's because I'm spawning the child cmds in the parent, but waitpid'ing in a child process of my own.
Why do that, you ask? First, if IPC::Run::start() and $ipc_run_h->finish() run in the same session, then all is well. This is what I did for regular cmds. With "pipe out" cmds like "|cmd1|cmd2|cmd3", I must pump data to the FH between calling start() and finish(). I also need to be able to process the output as it become available (rather than wait for the entire series to finish), since some cmds take a long time to finish. This is why I fork and have the child run $ipc_run_h->finish(), which leaves it in a different session.
Now that you know about the problem and goal, here's the complete code for illustrative purpose.
First, the main program, runPipe_PerlMonks.pl
# Uses RunPipe_PerlMonks module to run and trap failures in piped comm
+ands.
use strict;
use warnings;
use lib "/PATH_TO_RunPipe_PerlMonks";
use RunPipe_PerlMonks;
# Here we pass a series of piped cmds to be run by IPC::Run.
# The 3rd cmd, "grep zzzzzzzz", will fail. Goal is to identify it as t
+he cmd that made the series fail (a normal system call only gets the
+status of the last cmd, which in this case is a (false) success).
my @cmds = ( ['sort'], ['uniq', '-c'], [qw(grep zzzzzzzz)], ['sort', '
+-rnk1'] );
my $rp = new RunPipe_PerlMonks('cmds' => \@cmds);
# This will open the FH for the cmd, like "open FH_GLOB, '|-', 'sort |
+ uniq -c ...'", but it must be done by IPC::Run::start() since we're
+using it to run the cmds. Must be done before passing data to the FH.
$rp->start(\*FH_GLOB);
####### Now pump some data to the FH ######
my $max = 5;
for (my $i = 0; $i < $max; $i++)
{
print FH_GLOB int(rand($max)) . "\n";
}
close FH_GLOB;
#############################################
# Get the FH that will contain the output so we can process it in real
+ time (rather than waiting for everything to finish first).
my $fh = $rp->fh; # or die "Error: \$rp->fh not defined.\n";
# This forks. The parent returns to the main program (i.e. here) imme
+diately so we can process the results while waiting for them to finis
+h. The child calls $ipc_run_h->finish(), which waits for all cmds (i.
+e. children) to finish. It doesn't work because it's a child waiting
+for another child. Is there a better way to do this?
$rp->run();
while (my $line = <$fh>)
{
# do something with each output line.
print $line;
}
close $fh;
my $failed_cmd = RunPipe_PerlMonks::failed_cmd;
# Important: we want to identify the exact cmd that failed in a series
+ of piped cmds. A normal system call only returns the status of the l
+ast cmd. This is why we're using IPC::Run.
if (defined $failed_cmd)
{
print "Failed! cmd = '$failed_cmd'\n";
}
Here's RunPipe_PerlMonks.pm
package RunPipe_PerlMonks;
use strict; use warnings;
use IPC::Shareable; use IPC::Run; use FileHandle;
my $failed_cmd; my $glue = 'h_09';
my %options = (
create => 'yes',
exclusive => 1,
mode => 0644,
destroy => 1,
);
# STATIC method.
sub failed_cmd
{
IPC::Shareable->clean_up_all;
return $failed_cmd;
}
sub new
{
shift; my $self = { 'cmds' => undef, @_ };
defined $self->{'cmds'} or die "ERROR: didn't get CMD in construct
+or.";
IPC::Shareable->clean_up_all;
bless $self; return $self;
}
sub fh
{
my $self = shift;
return $self->{'readfh'};
}
# Input is an array of array references of commands. For example: ( [
+"ls", "-al"], ["grep", "-v", "something"] );
sub start
{
my $self = shift;
my $fh_glob_ref = shift;
my @cmds = @{$self->{'cmds'}} or die "Error: can't start() when CM
+DS not defined!\n";
my ($readfh, $writefh) = FileHandle::pipe;
$self->{'readfh'} = $readfh;
$self->{'writefh'} = $writefh;
my @ipcArray = (); my $ipc_run_h;
#pipe each command to the next
foreach my $cmd (@cmds)
{
push @ipcArray, $cmd;
if (defined $fh_glob_ref && @ipcArray == 1) # ('<pipe', $fh) o
+nly needs to be added to the first cmd. DO NOT added it to later cmd
+s (doing so causes only the last cmd to be run).
{
push @ipcArray, '<pipe', $fh_glob_ref;
}
push @ipcArray, "|";
}
pop @ipcArray; # get rid of the last "|"
# Capture both stdout and stderr in $writefh.
$ipc_run_h = IPC::Run::start(@ipcArray, '>', $writefh, "2>", $writ
+efh);
$self->{'h'} = $ipc_run_h;
}
sub run
{
my $self = shift;
my $ipc_run_h = $self->{'h'};
my $readfh = $self->{'readfh'};
my $writefh = $self->{'writefh'};
defined $readfh or die "Error: READFH not defined\n";
defined $writefh or die "Error: WRITEFH not defined\n";
defined $ipc_run_h or die "Error: ipc_run_h not defined\n";
eval { tie($failed_cmd, 'IPC::Shareable', $glue, { %options }) };
$@ and die "ERROR: GLUE already bound.\n";
# Run cmd as child process, so we can return FH immediately to the
+ running program and process the output in real time.
my $pid = fork();
if ($pid) # parent
{
my $child_pid = waitpid($pid, 0);
if ($child_pid == -1)
{
die "Child stopped with an error! ($!)\n";
}
elsif ($child_pid == 0) # child still running
{
die "ERROR: Child still running, but it should not have re
+turned until done...\n";
}
# if it gets here, child has finished. Close parent's copy of
+writeFH.
# note: DONT close the ReadFH. It needs to remain open for th
+e driver program; it will trigger EOF when writeFH is closed.
close $writefh;
return;
}
####### From here onward, it's the child running.
#******* Now call $ipc_run_h->finish(), which waitspid() each chil
+d.
#******* THIS is the failure point in the code because now we're r
+unning as the child, and we need to waitpid() for other children (eac
+h cmd in the series).
#******* Since a child can't waitpid another child, waitpid inside
+ $ipc_run_h->finish() always returns -1, the correct exit code is nev
+er returned, and we can't identify the exact cmd that failed.
if (!$ipc_run_h->finish()) #failure at some point
{
my $ctr = 0;
foreach my $cmd (@{$self->{'cmds'}})
{
#find the point where the failure occured, get the return
+value at that point
if($ipc_run_h->result($ctr) != 0){
my $returnval = $ipc_run_h->result($ctr);
# Set the failed command.
$failed_cmd = join(" ",@{$cmd});
last;
}
$ctr++;
}
}
close $readfh;
close $writefh;
exit;
}
1;
Running the above gives no output. Empty STDOUT is expected (since grep zzzzzz will have found nothing). But if working correctly, it would have printed out "grep zzzzzz" the failed cmd (I define fail as anything with a non-zero $?).
Suggestions?
One alternative I can think of is to dump the data into a temp file rather than passing it thru FH_GLOB. This would eradicate the need for "pipe out" cmds, before which all was well. But I suspect there may be better ways still.
|