I am not sure if you are set on the process you have right now (running the program in the background and then emailing it to the user) or not. But I have done some work with online reports hitting a MySQL database (not a ton, I am still a newb, so all of you more experienced folks please correct me if you feel I am wrong), and have found that a couple thousand records should not be a major problem to run on the fly and return the results to the screen (or excel, pdf, ... file). I have found that the longest part is usually rendering the information on the users screen.

A couple suggestions that may or may not help.
1. Make the SQL statement as fast a possible (you are probably thinking no S#$@). But what I mean is, perl can do a lot of the things you may be trying to do with SQL (like substr, decode(in some cases), uppercase, etc).
    a) Also it may be fast to actualy break you statement into multiple statements. So, instead of joining 10 tables do multiple selects...
2. Take a look at Excel::Template(which does not appear to be on cpan at the moment) (not sure if this is an ok alternative to your tab delimited files or not...)

We are currently set up with CGI::Application HTML::Template PDF::Template and Excel::Template for our reports (with some other stuff mixed in)

OK, that is all for now. I am not sure if this is even close to an option for you. But it may help at least start some ideas as an alternate direction to go.
S t r e b e n M ö n c h

In reply to Re: Perl CGI timeout by StrebenMönch
in thread Perl CGI timeout by jewang

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.