The code act as an information collector for a system administrator. It is initiated remotely via Net::SSH::Perl, and must therefore automatically manage the maximum allowed nr. of seconds that a command may run, or the amount of stdout or stderr that can be produced.

For example, it may not assume that "who failedlogin" will be just work, and prevent it from running for say longer than 60secs and 5000 lines of stdout/stderr.

-0xbeef

Use select to poll for available data on each of your filehandles, adding to a variable to keep track of the amount of output they've produced.

Here's how I might do it, using IPC::Open3 and IO::Select. You should add the waitpid/close from your example:

#!perl -w
use IPC::Open3;
use IO::Select;
use Symbol qw(gensym);
use strict;
my $cmd = shift;
my $g_timeout = shift || 5;
my $g_maxlines = shift || 10;
my @output;
my @err;
$| = 1;
runcmd(\$cmd, \@output, \@err);
print "output = \n";
print "  $_\n" for @output;
print "error = \n";
print "  $_\n" for @err;

sub runcmd {
  my ($cmdref, $outref, $errref) = @_;
  my ($childin, $childout, $childerr);
  $childerr = gensym;

  my $pid = open3($childin, $childout,  $childerr,  $cmd)
    or die "Cannot run cmd '$cmd': $!\n";

  my $select = IO::Select->new or die "Cannot create select object: $!
+\n";
  my @hold_output;
  for (($childout, $childerr)) {
    $select->add($_);
    $hold_output[fileno($_)] = [0, 0, ""]; # eof, lines, buffer;
  }

  $@ = undef;
  eval {
    local $SIG{ALRM} = sub { die "alarm\n" };
    my $deadline = $g_timeout + time;
    my $g_stop = 0;
    alarm($g_timeout + 1);
    while (!$g_stop && $select->count > 0) {
      $! = 0;
      my @ready = $select->can_read($deadline - time);
      if (!@ready) {
        $g_stop = 1;
      }
      for my $handle (@ready) {
        my $fno = fileno($handle);
        my $line;
        my $bytesread = sysread $handle, $line, 1024;
        if ($bytesread) {
          $hold_output[$fno][2] .= $line;
          $hold_output[$fno][1] += $line =~ y/\n/\n/;
          if ($hold_output[$fno][1] >= $g_maxlines) {
            $select->remove($handle);
          }
        } elsif ($!) {
          die "$!";
        } else {
          $hold_output[$fno][0] = 1; #EOF
          $select->remove($handle);
        }
      }
    }
    alarm(0);
  };
  if ($@) {
    # print STDERR "\$\@ = $@\n";
    die unless $@ eq "alarm\n";
  }

  # Note: lines may exceed $g_maxlines because of buffering
  # of output in the child process
  @$outref = split /\n/, $hold_output[fileno($childout)][2];
  @$errref = split /\n/, $hold_output[fileno($childerr)][2];
}
[download]

Okay, thanks for that! I would like however like to be enlightened more: ;)

1. I'd rather not use a solution using external modules, since the collector will have to run on MANY hosts as painlessly and non-intrusively as possible. Is there a high performance approach without external modules?
2. I am unsure as to how much data should be read from the pipe at a time in order to optimise throughput as much as possible...


-0xbeef

1. I'd rather not use a solution using external modules, since the collector will have to run on MANY hosts as painlessly and non-intrusively as possible. Is there a high performance approach without external modules?

The modules cited in the code that Thelonius posted (IPC::Open3, IO::Select, Symbol) are not "external", in the sense that they are all included in the standard Perl "core" distribution. That is, wherever a reasonably current version of Perl is installed, these modules are also installed by default.

(If these many hosts you speak of have non-standard or hopelessly outdated perl installations, that's going to be a problem anyway.)

As for buffer size, the 1024 bytes suggested by Thelonius is fine for handling text that must ultimately be treated in a line-oriented fashion; if you're dealing with really high data rates, an 8k buffer should be about optimal.

Ok, here is the code snippet. In a perl nutshell, the parent forks a child which does the actual o/s command, and the parent continues to monitor the child's duration and output size.

The child will redirect its output to the pipes and exec(). The parent _needs_ to acquire the child's stdout and stderr as fast as possible, and be able to handle (kill) a child running for longer than X time or larger than Y size.

   pipe READOUT,WRITEOUT;
   pipe READERR,WRITEERR;
   $g_pid = fork();
   if ( $g_pid < 0 ) {        # Error
      die "Error: fork failed with error $g_pid: $!.";
      $rc = 1;
   } elsif ( $g_pid == 0 ) {     # Child
      close(READOUT);
      close(READERR);
      close(STDOUT);
      open(STDOUT,">&WRITEOUT");
      close(STDERR);
      open(STDERR,">&WRITEERR");
      close(STDIN);
      open(STDIN,"</dev/null");
      exec($$cmdref);
      exit(1);
   } else {             # Parent
      $g_did_run++;
      $g_pids{$g_pid} = 1;
      close(WRITEOUT);
      close(WRITEERR);
      if ( $g_timeout > 0 ) {
         $g_stop = 0;
         alarm $g_timeout;
      }

      my $r_in = "";
      my $w_in = "";
      vec($r_in,fileno(READOUT),1) = 1;
      vec($r_in,fileno(READERR),1) = 1;
      vec($w_in,fileno(STDOUT),1) = 1;
      my $e_in = $r_in | $w_in;
      my ($r_out,$w_out,$e_out);
      my $cnt=0;
      my $eof_e = 0;
      my $eof_o = 0;
      while (1) {
         my ($nfound,$timeleft) = select($r_out=$r_in,undef,$e_out=$e_
+in,1);
         prog_log(20,sprintf("run_cmd: select returned time [%d] found
+ [$nfound] timeleft [$timeleft] r_out [%s] r_err [%s] e_out [%s] e_er
+r [%s]\n", time,
            vec($r_out,fileno(READOUT),1)?'y':'n',
            vec($r_out,fileno(READERR),1)?'y':'n',
            vec($e_out,fileno(READOUT),1)?'y':'n',
            vec($e_out,fileno(READERR),1)?'y':'n'));
         if ( vec($r_out,fileno(READERR),1) and !vec($e_out,fileno(REA
+DERR),1) ) {
            if ( eof(READERR) ) {
               $eof_e = 1;
            } else {
               my $line = <READERR>;
               chomp $line;
               push @$errref, $line;
            }
         }
         if ( vec($r_out,fileno(READOUT),1) and ! vec($e_out,fileno(RE
+ADOUT),1) ) {
            if ( eof(READOUT) ) {
               $eof_o = 1;
            } else {
               my $line = <READOUT>;
               chomp $line;
               push @$outref, $line;
            }
         }
         last if ( $eof_e and $eof_o );
         last if ( $g_stop );
      }
      if ( $g_stop ) { # Indicator for select loop to break
         $rc = 2;
      } else {
         my $loglvl = 5;
         my $t_pid = waitpid($g_pid,0);
         if ( $t_pid < 0 ) {
            $rc = 3;
         } elsif ( $t_pid == 0 ) {
            # Process no terminated yet!
            $loglvl = 1;
            $rc = 4;
         } elsif ( $t_pid == $g_pid ) {
            $rc = $?;
         } else {
            # Not possible!
            $loglvl = 1;
            $rc = 5;
         }
         close(READOUT);
         close(READERR);
         delete $g_pids{$g_pid};
         prog_log($loglvl,"run_cmd: waitpid($g_pid) returned $t_pid - 
+rc $rc.");
      }
[download]

Thanks in advance!
-0xbeef

Minor nit-pick that probably doesn't answer your immediate problem: "fork()" does not return a negative number; failure is indicated by returning undef. So your initial "if" condition should be:

  if ( ! defined( $g_pid ))
[download]

Apart from that, you seem to be using variables with global scope for flow-control, and you didn't show how/whether these were initialized ($g_stop, $g_timeout). So it's hard to guess whether it's a problem that $g_stop is never assigned a value within the "while(1)" loop. Also, you didn't include whatever code you have (if any) that actually handles the event generated by "alarm()". Could that be relevant?