Here are some thoughts I think apply.

One problem, you're using templating which requires that you build a very big data structure in memory, just so you can print it out.

The solution is to NOT build a giant data structure in memory, and print each row right after you retrieve it.

All of these variable seem redundant 

my ($key, %col_cbud, %col_encu, %col_fytd, %col_proj, %ctrl_tbl, @loop
+_data);

[download]
and I can tell without even seeing your template.

You might say but I need all those to smoosh the datasets together.

Well, thats what views are for. If you under utilize your database, you end up replicating a lot of its functionality. Databases are usually very good (well, good enough, depends on the data) at smooshing a bunch of datasets together.

In a recent thread, before I remembered about a special view called a pivot table, I ended up replicating the feature in perl, see Re: Open multiple file handles?. It took a while to write, a lot longer than it would to use a pivot table.

Leverage your database, study the sql, optimize it, its cheaper than reimplementing in perl. I realize its possible you may have good reasons for doing things in perl :)

Now the real trick to speeding up slow, long running programs, CGI or otherwise, is to refuse to run more than one (schedule only one) and cache the results.

For a technique see Watching long processes through CGI (Aug 02)

In reply to Re: Foreach loop takes a long time to process and my report times out before page is rendered by Anonymous Monk
in thread Foreach loop takes a long time to process and my report times out before page is rendered by Lori713

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.