C_T has asked for the wisdom of the Perl Monks concerning the following question:

I have a CGI which uses open3 to call a potentially very lengthy process (data processing). Since open3 doesn't return any output of the process until it completes, the problem I'm running into is that my CGI script spins for a while (~5 minutes?) then times out saying "could not open this page".

I know from experience that I can prevent this if I throw some stuff to the browser periodically, even if it's just a "." character. However I don't know if it's possible to do this while open3 is waiting around. Unfortunately, I can't just background the process and move on because what follows depends on the lengthly process being done.

It would be an easy problem to solve in a multi-threaded language (like Java). I would just create a thread to print "." to the browser every 15 seconds or so.

However, I'm not sure how to do that in Perl, or if it's even possible.

Does anyone have any thoughts to save my script? I'm open to any suggestions, not just a multi-threaded approach.

Thanks!

CT

Charles Thomas
Madison, WI
  • Comment on Long CGI process is cut short by browser

Replies are listed 'Best First'.
Re: Long CGI process is cut short by browser
by perrin (Chancellor) on Dec 02, 2004 at 23:04 UTC
    The basic approach for this sort of thing is the same as with threads, but you fork a process instead. A full example is here.
[reply]
      Ok, you're my hero!

      MANY THANKS!

      CT

      Charles Thomas
      Madison, WI
[reply]
      This example by Randal helped me figure out how to keep the process alive.

      Perhaps you guys can solve the other part of the equation. Right now I'm doing something like this: 

       #===== Open I/O pipes and run the process
  $pid = open3(\*DEV_NULL, \*OUT, \*ERR, "my_long_process");
  
  #===== Print the results
  while (<OUT>)
  {
    $buffer .= $_;
  
    #===== Update the display cache
    $cache->set($session, [0, $buffer]);
  }

      [download]

      Which works, except it waits until my_long_process completes before it sends any output back to the browser. If my_long_process takes an hour to run, the browser sits there for an hour with no updated information.

      Is there a way to set open3() to an unbuffered mode so it returns what my_long_process is doing in real time?

      Failing that, is there a better way to do this long process in the fork such that I can get its results as its doing things rather than waiting until the end?

      Thanks!

      CT

      Charles Thomas
      Madison, WI
[reply]
[d/l]
Re: Long CGI process is cut short by browser
by BUU (Prior) on Dec 03, 2004 at 03:57 UTC
    Just to correct you, Perl is a multi-threaded language. Or atleast, it can be. It depends on how it was built, but most perls these days seem to be threaded.
[reply]
Re: Long CGI process is cut short by browser
by dragonchild (Archbishop) on Dec 03, 2004 at 14:23 UTC
    While you have your answer, your question was in error. The browser does not cut off your process - the webserver does. You can also increase the TimeOut parameter to some large number, if needed.

    Being right, does not endow the right to be rude; politeness costs nothing.
    Being unknowing, is not the same as being stupid.
    Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
    Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

[reply]
      Actually, your reply is in error. Many browsers DO have an internal timeout value. IE's for example defaults to 5mins in v4.x, 60min in v5+, and can be adjusted thru a 'ReceiveTimeout' value in the registry.

      -Nitrox

[reply]
        I just did a search of my WinXP registry and didn't come across that key. Also, I have set my Apache TimeOut to 15minutes, let it run for 14, and had it work with both IE and Firefox.

        *shrugs*

        Being right, does not endow the right to be rude; politeness costs nothing.
        Being unknowing, is not the same as being stupid.
        Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
        Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

[reply]

Back to Seekers of Perl Wisdom