use Carp;
use IO::Handle;
use IPC::Open3;
use strict;

my $STDOUT = IO::Handle->new;
my $STDERR = IO::Handle->new;
open($STDOUT, ">&STDOUT") ||
    croak( 'Cannot duplicate STDOUT to file handle - ', $! );
open($STDERR, ">&STDERR") ||
    croak( 'Cannot duplicate STDERR to file handle - ', $! );

eval {
    open3(
        '<&STDIN',
        $STDOUT,
        $STDERR,
        "some_application"
    ) || die $!;
    waitpid(-1, 0);
};
croak( 'Cannot execute command - ', $@ ) if $@;

#   The STDOUT and STDERR output of some_application 
#   execution now resides in $STDOUT and $STDERR 
#   respectively. eg.

print $_ foreach <$STDOUT>
[download]

As always with Perl, TMTOWTDI.

This doesn't work either - if my program prints something to stdout and runs *forever* (ie. a daemon) then I will get no output. If I put while (<$STDOUT>) just after the open3, then everything is silent.

Unless my some_application itself sets $|=1 ... then I get output from it. If it's still buffered, I have to wait until the stdout buffer is full or the program terminates.

What I want is to read the stdout of some_application, buffered or no, in real time. The key question is becoming: 'why isn't some_application flushing its buffer on newline?'

Comments? Please? :)

--
Ash OS durbatulk, ash OS gimbatul,
Ash OS thrakatulk, agh burzum-ishi krimpatul!
Uzg-Microsoft-ishi amal fauthut burguuli.

Interesting ... One thing which I would be looking to try at this point would be to incorporate autoflush on the created duplicate handles prior to the open3 invocation. eg.

my $STDOUT = IO::Handle->new;
my $STDERR = IO::Handle->new;
open($STDOUT, ">&STDOUT") ||
    croak( 'Cannot duplicate STDOUT to file handle - ', $! );
open($STDERR, ">&STDERR") ||
    croak( 'Cannot duplicate STDERR to file handle - ', $! );

$STDOUT->autoflush;
$STDERR->autoflush;
[download]

If you still have no joy with this, you may have to incorporate a flush of the respective buffer prior to reading from it. eg. $STDOUT->flush. This may be necessary if some_application is actively buffering its output.

I'd be very interested to hear how you go with this and see if this resolves your problems with your specific application.

 

My guess is that you want to look at either IPC::Open3 or Expect.

With Expect, the new program will be on a tty, so it will be line buffered, however your stdout and stderr will be mixed together.

With IPC::Open3, the stdout and stderr will be seperated, but stdout won't be line buffered.

However, as it says in the IPC::Open3 docs, you can use IO::select to do non-blocking I/O on the resulting filehandles.

When STDOUT isn't a tty, it turns off line buffering. Hence my requirement to snarf STDOUT in real time was thwarted, until I figured the following out.

This code snipped forks and execs an application, using select to wait for available output on the stderr and stdout for the exec()ed application. The streams are not interleaved.

use IO::Pty;
use IO::Select;

my $pty = new IO::Pty;
pipe($readerr, $writeerr);

if (my $pid = fork)
{
    close($writeerr);
    my $select = new IO::Select;
    $select->add($pty);
    $select->add($readerr);
    while (1)
    {
        foreach my $fh ($select->can_read)
        {
            my $buf;
            if (sysread($fh, $buf, 4096))
            {
                print "Read ... $buf ...\n";
            }
        }
    }
}
else
{
    close($readerr);
    $pty->make_slave_controlling_terminal();
    my $slave = $pty->slave();
    close $pty;
    $slave->clone_winsize_from(\*STDIN);
    $slave->set_raw();
    open(STDOUT, ">&" . $slave->fileno);
    open(STDERR, ">&" . $writeerr->fileno);
    close($slave);
    exec("/home/hagus/foo.pl");
}
[download]

It turns out that non-line buffering when stdout isn't a tty is standard UNIX practice. So it's not a perl specific wrinkle. Try 'man stdio' on your system (solaris has a particularly detailed page on this).

So other options might have included using setvbuf under perl (POSIX::setvbuf?) to manually set the stdout descriptor to line buffered. Of course, that would be useless if the first thing that the exec()ed perl program did was test stdout for tty-ishness and revert to non-line buffered ...

HTH!

--
Ash OS durbatulk, ash OS gimbatul,
Ash OS thrakatulk, agh burzum-ishi krimpatul!
Uzg-Microsoft-ishi amal fauthut burguuli.

Any if those other options work - why?! I am guessing that underneath they're probably doing the same thing I am. The pragmatic me says 'try the other options', the pedantic me says 'yes, but *why* doesn't this current method work?'.

--
Ash OS durbatulk, ash OS gimbatul,
Ash OS thrakatulk, agh burzum-ishi krimpatul!
Uzg-Microsoft-ishi amal fauthut burguuli.

This may be a long shot, but *how* are you reading that output? are you using the diamond operator, or are you reading via select(IO::Select) + read/sysread ? It may not have anything to do with your problem at all, but who knows...

FWIW, something like the following works for me

# note: this code will go into infinite loop
# and is not a particularly good piece of code...
use IO::Handle;
use IO::Select;

my( $readfh, $writefh ) = ( IO::Handle->new(), IO::Handle->new() );

pipe( $readfh, $writefh );

if( my $pid = fork() ) {
    $writefh->close();
    my $select = IO::Select->new( $readfh );
    my $buf;
    while( 1 ) {
        if( $select->can_read( 1 ) ) {
            if( read( $readfh, $buf, 4096, 0 ) ) {
                print "got '$buf'";
            }
        }
    }
} else {
    $readfh->close;

    open( STDOUT, sprintf( '>&%d', $writefh->fileno ) );
    STDOUT->autoflush(1);

    exec( 'find', '/usr/local/lib/perl5' );
}
[download]

First, I changed 'read' to 'sysread'. But if the program you are exec()ing is this:

#!/usr/bin/perl

my $i=0;
while (1)
{
    print "stdout " . $i++ . "\n";
    sleep 1;
}
[download]

Then you will find that no output is captured by read. The perl program must set $|=1, which is exactly what I don't want to have to worry about.

--
Ash OS durbatulk, ash OS gimbatul,
Ash OS thrakatulk, agh burzum-ishi krimpatul!
Uzg-Microsoft-ishi amal fauthut burguuli.