You may also want to check out Synchronizing STDERR and STDOUT.

How can I get the STDOUT and STDERR from that pipe without waiting for it to complete ...

It sounds like you are asking how to get the output before it is produced? I'd like to know how they do that on unix!

If that's not what you are asking, then how about you clarify what you do mean?

I believe that what he means is to get some output out before the entire set of output is finished.
On unix you could do this with

[user@host]> streaming_output_script.pl&
[download]

Well, I doubt you need to do that on unix. I certainly don't need to on Win32.

Given junk2.pl that looks like this (note this script won't terminate naturally for nearly 3 hours):

#! perl -slw
use strict;

$|++;

for( 1 .. 10000 ) {
    print scalar localtime;
    sleep 1;
    warn "$0 still running\n";
}
[download]

And calling that from this script using a piped open in the same manner as the OP uses:

#! perl -slw
use strict;

my $pid = open my $fh, 'perl.exe junk2.pl |' or die $!;
while( <$fh> ) {
    chomp;
    print localtime() . " Got: '$_'";
}
[download]

I get this output:

c:\test>junk3
Sat Aug  4 02:33:24 2007 Got: 'Sat Aug  4 02:33:24 2007'
junk2.pl still running
Sat Aug  4 02:33:25 2007 Got: 'Sat Aug  4 02:33:25 2007'
junk2.pl still running
Sat Aug  4 02:33:26 2007 Got: 'Sat Aug  4 02:33:26 2007'
junk2.pl still running
Sat Aug  4 02:33:27 2007 Got: 'Sat Aug  4 02:33:27 2007'
junk2.pl still running
Sat Aug  4 02:33:28 2007 Got: 'Sat Aug  4 02:33:28 2007'
Terminating on signal SIGINT(2)

c:\test>junk2.pl still running
junk2.pl still running
junk2.pl still running
junk2.pl still running
junk2.pl still running
junk2.pl still running
...
[download]

Which shows that the parent receives the output from the child as soon as it produces it. And I seriously doubt that the '&' is required under unix either.

Even if I leave stdout buffering enabled

#! perl -slw
use strict;

#$|++;

for( 1 .. 10000 ) {
    print scalar localtime;
    sleep 1;
    warn "$0 still running\n";
}
[download]

It takes a while longer (~3 minutes) before the child fills the output buffer and the parent starts to receive the output (in one huge flurry):

c:\test>junk3
junk2.pl still running
junk2.pl still running
...
165 identical lines elided
...
junk2.pl still running
junk2.pl still running
Sat Aug  4 02:38:30 2007 Got: 'Sat Aug  4 02:35:51 2007'
Sat Aug  4 02:38:30 2007 Got: 'Sat Aug  4 02:35:52 2007'
...
156 monotonically increasing lines elided
...
Sat Aug  4 02:38:31 2007 Got: 'Sat Aug  4 02:38:28 2007'
Sat Aug  4 02:38:31 2007 Got: 'Sat Aug  4 02:38:29 2007'
junk2.pl still running
junk2.pl still running
...
[download]

But it certainly doesn't have to wait 3 hours for the child to finish before it gets it.

So, unless java uses a huge output buffer, or the java program produces very little output, I doubt the problem is down to buffering either. And if it is, there is nothing the OP could do with buffering in the calling script to change it anyway.

So, it comes back to the question, what exactly did the OP mean when he said:

How can I get the STDOUT and STDERR from that pipe without waiting for it to complete...

And until he clarifies that, his SoPW is unanswerable.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Ordinarily I'd recommend Watching long processes through CGI, but I'm not sure how well that will work on Win32.

I just had this same problem a few days ago with a C program. It turned out the C program was not flushing its buffer after each print statement.

Thanks! You were correct, the java program was not flushing after prints.

Auto-flushing the java program's print statements solved the problem.

My first post for help on PerlMonks has been answered!

I think that I'd have one page with a button on it to initiate the process in the background, and another page to take you to the output of the last run.

When you click the button, update the text for the output page to something like "Currently running, but here's the output of the previous run:", and disable the button. Then have a cron jobscheduled task periodically run and check the timestamp of that file. If it's new, start the test! When the test finishes, it simply resets the output page to the normal test output.

Just a thought...

...roboticus