in reply to Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file

I'd try the following: 

use IPC::Run qw( run new_chunker );

sub new_prefixer {
   my ($prefix) = @_;
   return sub {
      my ($in_ref, $out_ref) = @_;
      return input_avail && do {
         $$out_ref = join('', $$out_ref, $prefix, $$in_ref);
         $$in_ref = '';
         1
      };
   }
}

sub new_printer {
   my ($fh) = @_;
   return sub {
      print $fh ($_[0]);
   }
}

{
   my @cmd = ('find', '/fst/home/', '-name', '*ooo*');

   open(my $fh_log, '>>', 'h1.log')
      or die("Unable to open log file: $!\n");

   my $printer = new_printer($fh_log);

   run \@cmd,
      '>', \$fh_log,
      '2>', new_chunker(), new_prefixer('STDERR: '), $printer;
      or die("Error executing child. Child returned $&\n");

   # or:
   #
   # run \@cmd,
   #    '>',  new_chunker(), new_prefixer('STDOUT: '), $printer,
   #    '2>', new_chunker(), new_prefixer('STDERR: '), $printer;
   #    or die("Error executing child. Child returned $&\n");
}

[download]

Notes:

  • Comment on Re: Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file
  • Select or Download Code

Replies are listed 'Best First'.
Re^2: Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file
by Anonymous Monk on Apr 18, 2006 at 17:46 UTC
    Ikegami:

    We don't appear to have the IPC::Run module on our system, and if I can do this with open3, that's the way I'd like to go, because I see lots of "Use"s in the source for IPC::Run that I bet we also don't have.

    Thanks for your response.

[reply]
      I see lots of "Use"s in the source for IPC::Run that I bet we also don't have.

      Doubtful.
      CarpCore.
      ExporterCore.
      FcntlCore.
      File::BasenameCore.
      File::SpecCore.
      IO::HandleCore.
      IO::FileCore.
      IO::PtyConditionally required. If you need it for IPC::Run, you also need it for IPC::Open3.
      IO::TtyConditionally required. If you need it for IPC::Run, you also need it for IPC::Open3.
      IPC::Run::DebugComes with IPC::Run.
      IPC::Run::IOComes with IPC::Run.
      IPC::Run::TimerComes with IPC::Run.
      IPC::Run::Win32HelperComes with IPC::Run.
      IPC::Run::Win32IOComes with IPC::Run.
      IPC::Run::Win32PumpComes with IPC::Run.
      POSIXCore.
      strictCore.
      SocketCore.
      SymbolCore.
      UNIVERSALCore.
      varsCore.

      Not all of these are necessarily required or loaded.

      Feel free to write your own solution with IPC::Open3.

[reply]
Re^2: Prepending a string to STDERR output, and logging STDOUT & STDERR synchronously to a file
by OfficeLinebacker (Chaplain) on May 19, 2006 at 19:51 UTC
    Hey!

    So I got a version of IPC::Run installed on one of the machines here and I can't for the life of me get your program to work, ikegami. I get 

    /fst/home/m1jwc03/prod1/lib/cron% ./cron2.pl /fst/prod1/hedgefunds/Lipper/test.pl m1jwc03
Bareword found in conditional at ./cron2.pl line 46.
Use of uninitialized value in scalar dereference at ./cron2.pl line 42.
Use of uninitialized value in join or string at ./cron2.pl line 42.
Use of uninitialized value in join or string at ./cron2.pl line 42.
Use of uninitialized value in scalar dereference at ./cron2.pl line 42.
Use of uninitialized value in join or string at ./cron2.pl line 42.
Use of uninitialized value in scalar dereference at ./cron2.pl line 42.
Use of uninitialized value in join or string at ./cron2.pl line 42.
Use of uninitialized value in scalar dereference at ./cron2.pl line 42.
Use of uninitialized value in join or string at ./cron2.pl line 42.
Use of uninitialized value in scalar dereference at ./cron2.pl line 42.
Use of uninitialized value in join or string at ./cron2.pl line 42.
    line 42 is 
    $$out_ref = join('', $$out_ref, $prefix, $$in_ref);

    [download]
    and line 46 is four lines down--the brace that closes the sub definition in the line 'return sub{' in new_prefixer.

    TIA,

    T _________________________________________________________________________________
    Without me, it's just aweso

[reply]
[d/l]

      Change
      use IPC::Run qw( run new_chunker );
      to
      use IPC::Run qw( run new_chunker input_avail );

      I don't know if that will fix everything, but it's a start. Let me know.

[reply]
[d/l]
[select]
        Hey guy.

        While I was waiting I tried a few things and am up to here: 

        #!/opt/local/bin/perl -w
use IPC::Run qw( run new_chunker );
use File::Basename;
#need two arguments: script to run and mailing list (comma-separated m
+1 ids)
$#ARGV == 1 || die "Error: Need exactly two arguments!\n";
($base, $dir, $ext)=fileparse($ARGV[0],'\..*');
#Do some machinations to create a unique overall log file
@now=localtime;
#month returned by localtime() are zero-indexed.
#got to pad the zeroes for the time/date elements
$mo=sprintf("%02d",$now[4]+1);
$d=sprintf("%02d",$now[3]);
$y=$now[5]+1900;
$h=sprintf("%02d",$now[2]);
$mi=sprintf("%02d",$now[1]);
$workdir="/fma/prod1/isa/logs";
$logfile="$workdir/$base-$y$mo$d.$h$mi.m1jwc03.log";
open(LOG,">$logfile") || die "Error:  Could not open $logfile: $!\n";
my $oldfh=select(LOG) || die "Error:  Could not select LOG: $!\n";
print "*-*-begin_run-*-*\n";
($distlist = $ARGV[1]) =~ s/,/ /g;
print "*-*-on_error $distlist-*-*\n";
chdir("$dir");

@cmd=$ARGV[0];

$SIG{CHLD} = sub {
   print "NOTE:  in sig{CHLD}\n";
#   if (waitpid($pid, 0) > 0){sleep(1);};
};
{
   run (\@cmd, '>pty>', sub{ print "STDOUT: $_[0]";}, '2>', sub { prin
+t "STDERR: $_[0]";}) or die("Error executing child. Child returned $&
+\n");

}
print "*-*-end_run-*-*\n";
close(LOG);

        [download]
        Which is SOOO close. The stuff all comes out in order, but I think the print function is getting some STDERR messages in chinks. However I don't mind TOO much because all I REALLY want to do is prepend and append html tags. So is there a reason why I couldn't turn the run statment into 
        run (\@cmd, '>pty>', sub{ print "$_[0]";}, '2>', sub { print "<b><big>
+STDERR: $_[0]</b></big>";}) or die("Error executing child. Child retu
+rned $&\n");

        [download]
        ?

        ________________________________________________________________________________
        Without me, it's just aweso

[reply]
[d/l]
[select]
        Hey dude(s)

        Also in the meantime, I have tried 

        #!/opt/local/bin/perl -w
use IPC::Run qw( run new_chunker input_avail );
use File::Basename;
#need two arguments: script to run and mailing list (comma-separated m
+1 ids)
$#ARGV == 1 || die "Error: Need exactly two arguments!\n";
($base, $dir, $ext)=fileparse($ARGV[0],'\..*');
#Do some machinations to create a unique overall log file
@now=localtime;
#month returned by localtime() are zero-indexed.
#got to pad the zeroes for the time/date elements
$mo=sprintf("%02d",$now[4]+1);
$d=sprintf("%02d",$now[3]);
$y=$now[5]+1900;
$h=sprintf("%02d",$now[2]);
$mi=sprintf("%02d",$now[1]);
$workdir="/fma/prod1/isa/logs";
$logfile="$workdir/$base-$y$mo$d.$h$mi.m1jwc03.log";
open(LOG,">$logfile") || die "Error:  Could not open $logfile: $!\n";
my $oldfh=select(LOG) || die "Error:  Could not select LOG: $!\n";
print "*-*-begin_run-*-*\n";
($distlist = $ARGV[1]) =~ s/,/ /g;
print "*-*-on_error $distlist-*-*\n";
chdir("$dir");

@cmd=$ARGV[0];

$SIG{CHLD} = sub {
   print "NOTE:  in sig{CHLD}\n";
#   if (waitpid($pid, 0) > 0){sleep(1);};
};
sub new_prefixer {
   my ($prefix) = @_;
   return sub {
      my ($in_ref, $out_ref) = @_;
      return input_avail && do {
         $$out_ref = join('', $$out_ref, $prefix, $$in_ref);
         $$in_ref = '';
         1
      };
   }
}
sub new_printer {
   if (defined ($_[0])){
     return sub {
       print ($_[0]);
     }
   }else{
     return sub {
       print '$_[0] undefined in new_printer',"\n";
     }
   }
}
{
   my $printer = new_printer();

   run (\@cmd, '>', \*LOG, '2>', new_chunker(), new_prefixer('STDERR: 
+'), $printer) or die("Error executing child. Child returned $&\n");
}
print "*-*-end_run-*-*\n";
close(LOG);

        [download]
        Which doesn't give me any nasty output in the shell, but the log only gets STDOUT and after the STDOUT there are a series of lines that look like this: 
        $_[0] undefined in new_printer
$_[0] undefined in new_printer
$_[0] undefined in new_printer
.....
$_[0] undefined in new_printer
$_[0] undefined in new_printer
NOTE:  in sig{CHLD}
$_[0] undefined in new_printer
$_[0] undefined in new_printer
*-*-end_run-*-*
        so for some reason new_printer() is not getting the right data?

        Any and all comments appreciated, and I hope you don't mind that I am taking liberties with your code (to simplify and adapt to my needs).

        Thanks again.

        _________________________________________________________________________________
        Without me, it's just aweso

[reply]
[d/l]
        trying it like this: 
         
run (\@cmd, '>pty>', new_chunker(), sub{ print "STDOUT: $_[0]";}, '2>'
+, new_chunker(), sub { print "STDERR: $_[0]";});

        [download]
        ALMOST works, but some lagging STDERR comes out at the end.

        _________________________________________________________________________________
        Without me, it's just aweso

[reply]
[d/l]

In Section Seekers of Perl Wisdom