One industrial-strength way is to fork off a child which does the processing while the parent keeps the browser from timing out by printing spaces, periods, or intermittent status messages. You need to have the parent set output autoflush on ($|=1). You also could do it without a child but use alarms, as mentioned in the timeout discussion mentioned above.

If you can get the child to send messages to the parent thread during processing, those intermittent status messages could be more interesting. I am thinking of doing this for a similar problem we talked about recently at the monastery.. some message passing pipe or possibly IPC may be useful for this.

One idea, if you are going to have a ton of files to be processed maybe you want to have one server which just searches all these files, doing the optimization, scheduling, and sorting you need done, and have cgi processes talk to the server. That way you might be able to allot more cpu to the processing daemon. But you might get similar timeout issues.