Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I am looking for the best structure to create to manipulate some data.
I have several SQL statements in @statements to execute. I use the fetchrow_hashref to get rows. (The results of the previous statement will be used to execute the next request). The first statement returns only one row (hash_ref), and the others one can return one or many.
So, I need to access easily in this structure the field name (key of the hash_ref), the value and the number of rows sent by the statement.
I would like to handle all the results in a unique structure. I am thinking to this kind of hash
my %struture = ( '1' => { 'field1_statement1' => 'value', 'field2_statement1' => 'value' }, '2.1' => { 'field1_statement2' => 'value', 'field2_statement2' => 'value', 'field3_statement2' => 'value' }, '2.2' => { 'field1_statement2' => 'value', 'field2_statement2' => '', 'field3_statement2' => 'value' }, '3.1' => { 'field1_statement2' => 'value' } );
and this is the code to generate it
my @statements; my %structure; foreach my $elt ( @statements ) { my @ex = exec($elt); my $i; for (my $y=0;$y<=$#ex;$y++) { my %results_temp = %{$ex[$y]}; my $index = "$i.$y"; foreach my $key (keys %results_temp) { $structure{$index}{$key} = $results_temp{$key}; } } } sub exec { my @param = @_; my ($type, $db, $login, $pwd, $ip) = getDBInfo(); my $dbh = connectDB ($type, $db, $login, $pwd, $ip); my @store; foreach (@param) { my $sth = $dbh->prepare ($_); $sth->execute() || die $sth->errstr; while (my $hash_ref = $sth->fetchrow_hashref) { push @store, $hash_ref; } } return @store; }
What do you think? Is it relevant?
I don’t think it is efficient to count the number of 2.x. Either I calculate it on the fly or I store it in the structure, I don’t know…
Obviuously, I need an index to not override data.
Thanks in advance

Replies are listed 'Best First'.
Re: Best complex structure?
by merlyn (Sage) on Jun 08, 2005 at 14:20 UTC
      You are right, thanks. I changed the name. It is sometimes difficult to find a accurate method/variable name.
Re: Best complex structure?
by BrowserUk (Patriarch) on Jun 08, 2005 at 17:34 UTC

    A couple of thoughts. Your top-level keynames appear to serve two purposes:

    1. ordering your data;

      But this requires you to sort the hash to recover the ordering.

    2. keeping a count of the number of values returned by your 3 queries;

      But it forces you to sort and count the values Which you've recognised by considering accumulating the counts within separate hash keys.

    A better solution to both problems is to use arrays. Actually an Array of Arrays (of Arrays) of Hashes.

    @structure = ( [ # Result from first query. { ... }, { ... }, ], [ Results from second query [ # first parameter { ... } { ... } ], [ # second parameter { ... }, { ... } ] ], [ # results from third query [ # first parameter { ... } ], ... ] );

    You may be able to flatten the inner levels of the last two query results -- it's not clear from your post whether you need this separation.

    This way, you neither need to sort to recover the ordering nor count to discover the numbers. The ordering is inherent and the counts are mantained by the individual arrays. The worst you might need to do is accumulate the total counts for a composite query:

    # Total results from all parameters of query 2 # See List::Util qw[ sum ]; my $q2Total = sum @{ $structure[ 1 ] };

    The second thought is that your using fetchrow_hashref in a tight loop and pushing to an array.

    From the DBI pod:

    If $slice is a hash reference, fetchall_arrayref uses fetchrow_hashref to fetch each row as a hash reference. If the $slice hash is empty then fetchrow_hashref() is simply called in a tight loop and the keys in the hashes have whatever name lettercase is returned by default from fetchrow_hashref. (See FetchHashKeyName attribute.) I

    Using the correct varient and parameters of the fetchall_arrayref will probably run more quickly and avoid a lot of copying.

    Lastly, without knowing what volumes or the nature of the data your pulling in, this maybe off target, but unless you absolutely need to have access to all this data simultaneously, and the processing that requires it is such that it cannot be done by the database itself, you probably ought to consider fetching that subset of the data you need at any given time rather than fetching it all in one big hit and storing it locally. Especially so if this is a (non-mod_perl) CGI program where your going to have to reload it for each request.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

      I would definitely go with BrowserUK's idea. Since you are numbering anyway, an array is perfect for you. Basically, the "3.1" is only important because it comes after "2.*" and before "4.*", so there's really no reason to put it in a hash, but rather an array is good.

      BrowserUK: Does your example have one to many levels of anonomous arrays? I would think it would more be something like

      @structure = ( { ... }, [ { ... }, { ... }, ], [ { ... }, { ... }, ], [ { ... }, { ... }, ], );

      That way you can do something like:

      my @structure = ( $hashref_from_first ); foreach @query { ... push @structure, [] foreach @subquery { ... push @structure->[0], $hashref; } }

      Or something like that (but probably a lot better)

          -Bryan

        As I mentioned in the post, it wasn;t clear to me if the OP needed to retain the 2.1/2.2/2.3 etc. levels arising from the multiple executions of the second and third statements with different bound parameters. Hence I added an extra level to retain them and noted that they could be removed if that information (count by parameter) was not required.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.