I'm not a cgi person, but there are various things that could be causing the symptoms you are describing. You need to determine where the delays are occuring.

If the java program produces a steady stream of output when invoked via the command line, it probably isn't the source of your problem.

You could confirm that it isn't your perl cgi script, by invoking it directly from the command line and watching what happens. If it also produces a steady stream when invoked this way, then there are a couple of other places where the problem could lie. Eg.

• Depending upon the browser involved, and the html you are outputting, the browser could be refusing to format and display what it has received, until it gets the end tag for one or more elements.

  Some (maybe all?) browsers refuse to format tables until they know how many or how big the rows are.

  You could probably verify if this is the problem, by temporarily switching the content type header to text/plain. That should cause the browser to display the html and content as it arrives rather than waiting until it knows how to format it.

  If this is the problem, you may be able to avoid it by switching to using <p> or <br> formatting rather than (for example) a table.

• Depending upon the web server you are using, it may be buffering the output until it detects the end of the cgi task.

  A quick way of checking this would be to run the cgi script to completion, from the command line, having redirected stdout to a file. Then substitute a simple cgi script that slurps the file in one chunk, prints it to the browser in one chunk, and then sleeps for 5 minutes before exiting.

  If you see that output in the browser straight away, then the delay is in the browser. If you don't see the output for 5 minutes, the delay is in the server/cgi interface.

If you supply a better picture of what you are doing--what server, browser, html formatting etc--you are using, then one of the many experienced CGI people here will be probably be able to offer you better ways of determining the source of your problem, and probably a solution to it.

Ultimately, unless your web server is dedicated to running just this task, or just a few low-demand tasks, it's generally not a good idea to have a cgi script that takes hours to run. The usual solutions to this involve forking, but they generally don't work under Win32 unless you are also using cygwin. I do have an alternative to that, but you would still need to fix the above problem(s) first.

OK, Apache 2.2 server, either Mozilla or IE browser, and as far as the html formatting that goes on, the page in question will print a very simple header, then its basically this loop, from the OP:

$| = 1;
open(JAVA, "java_command_stuff |") or die ...;
while(<JAVA>){
   my $line = $_;
   print "$line<br>\n";
}
close JAVA;
[download]

After which I close out the html tags and the script ends.

I tried switching the header to text/plain but still don't get any output until task is complete.

Just an FYI, most all of the tests this runs, will complete in just a matter of seconds. There is just one pesky test case here that is running for so long, but I would like to be able to handle more 'long duration' cases in the future so I'm trying to work this out now...

thanks

I tried switching the header to text/plain but still don't get any output until task is complete.

It seems clear that the problem lies within the server/cgi interface. Ie. Apache is buffering the output until the perlscript completes? I don't use Apache so I can't confirm that. Did you try putting the output into a file and using a slurp'n'spit script with a final sleep to confirm that?

My solution to the problem of dealing with long running cgis is to spawn a new process detached from the webserver that opens a port, redirects the client to that port and then takes over managing the http comms for that client instance until the task is complete.

Essentially putting a http daemon into the script and taking the webserver completly out of the loop. At that point you have complete control and can choose to either just present the output as it is produced, or present a html status page that auto refreshes periodically until the output is done.

The nice thing about this solution is that if the client stops refreshing--gets bored with waiting and closes the page, then you can abort the long running task early and prevent it from tying up resources. You can also ensure that only one copy of a long running task is started for any given set of parameters.

However, I am reluctant to offer this solution as my inexperience with things cgi in general, and things Apache in particular, mean that there is quite likely a simpler fix already known for Apache.

I would strongly recommend that you re-ask your question in a new SoPW, referencing this one, but omitting any reference to Windows in the title of the new one. You should explicitly reference 'Apache 2.2' and 'long running CGI' in the title. Ie. "Buffering problem with Apache 2.2 and a long running CGI". That is more likely to receive attention from the Apache and web experts here abouts who tend to use *nix and skip windows questions.

I would also suggest that you confirm that you encounter the same problem using a long running Perl script in place of the java one. Something simple like

#!perl -w
use strict;

$| = 1;
for( 1 .. 10,000 ) {
    print "$_: " . localtime;
    sleep 1;
}
[download]

before you re-post the question, and assuming that still demonstrates it, use that in place of the java script. It just removes one more unknown and means that those inclined to do so, will be able to reproduce your problem.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."