use IO::Handle;
 :
open3( $input, $output, '', $command );
$output->autoflush()
[download]

Simply doing $|++ doesn't help me in this case. Nor does setting autoflush(). I suppose I should be using getc() but that does not seem like a very perl way of doing things. I was hoping for something more interesting like setting the end of line delimiter to a regex but I don't see a way of doing this. I wouldn't want to create my own buffer for assembling characters back into strings. Any thoughts?

Two possibilities for you.

One is that the message that you are trying to capture may be sent on STDERR, not STDOUT. If so then you are just listening on the wrong handle.

The other is that the buffering is not happening in Perl, but in tar. If that is the case, then you could use something like IO::Pty to fool tar into thinking that it is talking to a terminal, and not on a pipeline.

use IO::Select;
{
  # Actually, I should have one buffer per filehandle, but
  # for simplicity, I'll assume you only ever call
  # this function with one filehandle.
  my $buffer = "";

  # Note that timeout is for waiting for the \n after the
  # first character arrives.  This function will block
  # forever waiting for the first character of the line
  sub readWithTimeout
  {
    my ($inputh, $timeout) = @_;
    if (length($buffer)==0) {
      sysread $inputh,$buffer,500;
      if (length($buffer)==0) {return undef;}
    }
    if ($buffer =~ s{^(.*\n)}{}) {return $1;}
    my $s = IO::Select->new();
    $s->add($inputh);
    my @ready = $s->can_read($timeout);
    while (@ready)
    {
      last unless
        sysread $inputh,$buffer,500,length($buffer);
      if ($buffer =~ s{^(.*\n)}{}) {return $1;}
      @ready = $s->can_read($timeout);
    }
    $buffer =~ s/.+// or return undef;
    return $&;
  }
}
[download]

But this being perl, there's another way to do it - and this is even reuseable. "Just" create a descendant of Tie::Handle that wraps an existing filehandle to provide this kind of timeout:

package TimeOutHandle;
use Tie::Handle;
use IO::Select;
@ISA = qw(Tie::Handle);
sub TIEHANDLE 
{
  my ($class, $wrappedfh, $timeout) = @_;
  my ($buffer) = '';
  $timeout ||= 5;  # default five second timeout
  return bless [$wrappedfh, $timeout, \$buffer];
}
sub READLINE
{
  my $self = shift;
  my ($inputh, $timeout, $bufref) = @$self;
  if (length($$bufref)==0) {
    sysread $inputh,$$bufref,500;
    if (length($$bufref)==0) {return undef;}
  }
  if ($$bufref =~ s{^.*\n}{}) {return $&;}
  my $s = IO::Select->new();
  $s->add($inputh);
  my @ready = $s->can_read($timeout);
  while (@ready)
  {
    last unless 
      sysread $inputh,$$bufref,500,length($$bufref);
    if ($$bufref =~ s{^.*\n}{}) {return $&;}
    @ready = $s->can_read($timeout);
  }
  $$bufref =~ s/.+// or return undef;
  return $&;
}

package main;
# Test code for the above.

# feed me some data slowly
sub slowdata()
{
  print "a"; sleep 3; print "b"; sleep 6;
  print "c\nd"; sleep 2; print "e";
}

# given a filehandle, tell me what <> does.
# Also, give me when <> does it.
sub reportlines
{
  my $fh = shift;
  printf "%02d: __BEGIN__\n", time() % 100;
  while (<$fh>)
  { s/\n/\\n/s; printf "%02d: '$_'\n", time() % 100; }
  printf "%02d: __END__\n", time() % 100;
}

$|=1;
print "Regular filehandle:\n";
open (INDATA, '-|') || do {slowdata();exit();};
reportlines(*INDATA);

print "\n5 second timeout:\n";
open (INDATA, '-|') || do {slowdata();exit();};
tie *TIMEOUT5, 'TimeOutHandle', \*INDATA;
reportlines(*TIMEOUT5);

print "\n1 second timeout:\n";
open (INDATA, '-|') || do {slowdata();exit();};
tie *TIMEOUT1, 'TimeOutHandle', \*INDATA, 1;
reportlines(*TIMEOUT1);

print "\n10 second timeout:\n";
open (INDATA, '-|') || do {slowdata();exit();};
tie *TIMEOUT10, 'TimeOutHandle', \*INDATA, 10;
reportlines(*TIMEOUT10);
[download]

Regular filehandle:
19: __BEGIN__
28: 'abc\n'
30: 'de'
30: __END__

5 second timeout:
30: __BEGIN__
38: 'ab'
39: 'c\n'
41: 'de'
41: __END__

1 second timeout:
41: __BEGIN__
42: 'a'
45: 'b'
50: 'c\n'
51: 'd'
52: 'e'
52: __END__

10 second timeout:
52: __BEGIN__
61: 'abc\n'
63: 'de'
63: __END__
[download]

Re-implementing this as a subclass of IO::Handle instead of a tied filehash is left as an exercise for the reader.

#!/usr/bin/perl

# sgifford of perlmonks 
# The reason for your problem is because you're not using  
# IO::Select quite right. You're passing a timeout of 0 to  
# can_read, which asks it to return immediately if there is  
# no data ready yet. What I think you want to do is create  
# one IO::Select object with both handles, then ask it to  
# wait until one or both have something ready to read.  

# Something like this: 
# It's only drawback is it only outputs 1 line of bc output 
# so it errs on something like 234^12345 (which outputs a big number) 

use warnings;
use strict;
use IPC::Open3;
use IO::Select;

#interface to "bc" calculator 
my $pid = open3(\*WRITE, \*READ,\*ERROR,"bc");

my $sel = new IO::Select();

$sel->add(\*READ);
$sel->add(\*ERROR);

my($error,$answer)=('','');

while(1){

 print "Enter expression for bc, i.e. 2 + 2\n";
 chomp(my $query = <STDIN>);

 #send query to bc 
 print WRITE "$query\n";

 foreach my $h ($sel->can_read)
 {
   my $buf = '';
   if ($h eq \*ERROR)
   {
     sysread(ERROR,$buf,4096);
     if($buf){print "ERROR-> $buf\n"}
   }
   else
   {
     sysread(READ,$buf,4096);
     if($buf){print "$query = $buf\n"}
   }
 }
}

waitpid($pid, 1);
# It is important to waitpid on your child process,   
# otherwise zombies could be created.
[download]