One industrial-strength way is to fork off a child which does the processing while the parent keeps the browser from timing out by printing spaces, periods, or intermittent status messages. You need to have the parent set output autoflush on ($|=1). You also could do it without a child but use alarms, as mentioned in the timeout discussion mentioned above.

If you can get the child to send messages to the parent thread during processing, those intermittent status messages could be more interesting. I am thinking of doing this for a similar problem we talked about recently at the monastery.. some message passing pipe or possibly IPC may be useful for this.

One idea, if you are going to have a ton of files to be processed maybe you want to have one server which just searches all these files, doing the optimization, scheduling, and sorting you need done, and have cgi processes talk to the server. That way you might be able to allot more cpu to the processing daemon. But you might get similar timeout issues.

In reply to Re: Searching large files before browser timeout by mattr
in thread Searching large files before browser timeout by aijin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.