A couple of thoughts. Your top-level keynames appear to serve two purposes:

  1. ordering your data;

    But this requires you to sort the hash to recover the ordering.

  2. keeping a count of the number of values returned by your 3 queries;

    But it forces you to sort and count the values Which you've recognised by considering accumulating the counts within separate hash keys.

A better solution to both problems is to use arrays. Actually an Array of Arrays (of Arrays) of Hashes.

@structure = ( [ # Result from first query. { ... }, { ... }, ], [ Results from second query [ # first parameter { ... } { ... } ], [ # second parameter { ... }, { ... } ] ], [ # results from third query [ # first parameter { ... } ], ... ] );

You may be able to flatten the inner levels of the last two query results -- it's not clear from your post whether you need this separation.

This way, you neither need to sort to recover the ordering nor count to discover the numbers. The ordering is inherent and the counts are mantained by the individual arrays. The worst you might need to do is accumulate the total counts for a composite query:

# Total results from all parameters of query 2 # See List::Util qw[ sum ]; my $q2Total = sum @{ $structure[ 1 ] };

The second thought is that your using fetchrow_hashref in a tight loop and pushing to an array.

From the DBI pod:

If $slice is a hash reference, fetchall_arrayref uses fetchrow_hashref to fetch each row as a hash reference. If the $slice hash is empty then fetchrow_hashref() is simply called in a tight loop and the keys in the hashes have whatever name lettercase is returned by default from fetchrow_hashref. (See FetchHashKeyName attribute.) I

Using the correct varient and parameters of the fetchall_arrayref will probably run more quickly and avoid a lot of copying.

Lastly, without knowing what volumes or the nature of the data your pulling in, this maybe off target, but unless you absolutely need to have access to all this data simultaneously, and the processing that requires it is such that it cannot be done by the database itself, you probably ought to consider fetching that subset of the data you need at any given time rather than fetching it all in one big hit and storing it locally. Especially so if this is a (non-mod_perl) CGI program where your going to have to reload it for each request.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

In reply to Re: Best complex structure? by BrowserUk
in thread Best complex structure? by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.