rocky13 has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I have queried results from a database as hash reference. Now, I wish to put these results in a data structure. The results are in order based on the first three columns. I am trying to get the following structure:

Joe=>{ A=>{ 1=> { [85,80],[90,99] } 2=> { [50,60,87],[70,65,89] } } B=>{ 1=> { [82],[92] } 3=> { [30],[51] } } } Rob => continue as above
Basically, I need to do calculations on the last two columns: Joe,A,1's would get calculated, Joe,A,2's would get calculated,Joe,B,1's would get calculated etc. Please let me know if there is a better approach. I was having trouble separating the rows Joe,A,1's from Joe,A,2's and therefore i was not able to do any calculations. If there is a way just to separate the rows based on the first three columns, then I know how to do the rest. Thanks in advance! Here is the code with the subset of data I queried:

#!/usr/bin/perl use DBI; use DBD::mysql; print "Connecting...\n"; my $platform = "mysql"; my $database = "db_name"; my $host = "host_name"; my $port = "port_num"; my $tablename = "hist"; my $user = "user_name"; my $pwd = "pass"; open(STDOUT, ">C:\\perlscripts\\new.txt") || die "Can't open the file" +; my $dsn = "dbi:mysql:$database:$host:$port"; my $dbh = DBI->connect($dsn,$user,$pwd) || die "Could not connect: $DB +I::errstr\n"; my $query = $dbh->selectall_arrayref("select e_id,cus,ta,gd1,gd2 from +hist order by e_id,cus,ta", {Slice => {} }); foreach my $ref (@$query) { print "$ref->{e_id},$ref->{cus},$ref->{ta},$ref->{gd1},$ref->{gd2}\ +n"; } <p> Here is the subset of data I get from this code. </p> Joe,A,1,85,90 Joe,A,1,80,99 Joe,A,2,50,70 Joe,A,2,60,65 Joe,A,2,87,89 Joe,B,1,82,92 Joe,B,3,30,51 Rob,A,1,64,77 Rob,B,2,20,32

At the end, I want to be able to calculate the 4th and 5th column like this:

Joe,A,1,85,90 Joe,A,1,80,99 Total 165,189 Joe,A,2,50,70 Joe,A,2,60,65 Joe,A,2,87,89 Total 147,154 Joe,B,1,82,92 Total 82,92 Joe,B,3,30,51 Total 30,51 Rob,A,1,64,77 Total 64,77 Rob,B,2,20,32 Total 20,32

Replies are listed 'Best First'.
Re: hoh or hoa data structure
by Generoso (Prior) on Jan 11, 2011 at 00:08 UTC

      Yes, I have. You can use order by e_id, cus, ta, compute sum(col4), sum(col5) by e_id, cus, ta. I want to know if you can do it in Perl. I know there is a way to achieve the shown data structure because I did it once before with a stock file. Unfortunately, I can't figure it out this time and I can't find the code.

        I know there is a way to achieve the shown data structure because I did it once before with a stock file.

        Perhaps How can I visualize my complex data structure? will help you with your data structures; it's what I use to help me with mine.

        Unfortunately, I can't figure it out this time and I can't find the code.

        Oh, I really do recommend some form or fashion of a Version Control System. I am constantly looking back at things (even very broken things) to find out how I accomplished a particular task, or how not to try to solve a given problem. Recently, I've even begun putting various notes (usually OpenOffice docs) in because, in my case, TortoiseSVN makes it so very easy to keep things in synch between my main work machine and the netbook I carry whenever I leave my home office.

        HTH,

        planetscape
Re: hoh or hoa data structure
by GrandFather (Saint) on Jan 10, 2011 at 23:25 UTC

    Show us some code. Ideally a trivial database using something like the following as a starting point:

    #!/usr/bin/perl use strict; use warnings; use DBI; unlink 'results'; open my $resFile, '>', 'results'; print $resFile <<DATA; Joe,A,1,85,90 Joe,A,1,80,99 Joe,A,2,50,70 Joe,A,2,60,65 Joe,A,2,87,89 Joe,B,1,82,92 Joe,B,3,30,51 Rob,A,1,85,90 Rob,A,1,80,99 Rob,A,2,50,70 Rob,A,2,60,65 Rob,A,2,87,89 Rob,B,1,82,92 Rob,B,3,30,51 DATA close $resFile; my $dbh = DBI->connect("DBI:CSV:") or die $DBI::errstr; my $sth = $dbh->prepare("SELECT * FROM results"); $sth->execute(); my $arrayRef = $sth->fetchall_arrayref; print +(join ', ', @$_), "\n" for @$arrayRef; exit;

    Prints:

    Joe, A, 1, 80, 99 Joe, A, 2, 50, 70 Joe, A, 2, 60, 65 Joe, A, 2, 87, 89 Joe, B, 1, 82, 92 Joe, B, 3, 30, 51 Rob, A, 1, 85, 90 Rob, A, 1, 80, 99 Rob, A, 2, 50, 70 Rob, A, 2, 60, 65 Rob, A, 2, 87, 89 Rob, B, 1, 82, 92 Rob, B, 3, 30, 51
    True laziness is hard work

      I have updated the question with all of the code that I have. I am able to get the data results (5 columns) as shown but I cannot figure out a way to calculate the 4th and 5th column once ordered by the first three columns.

        Something like:

        #!/usr/bin/perl use strict; use warnings; use DBI; my $tableName = 'hist'; unlink $tableName; open my $resFile, '>', $tableName; print $resFile <<DATA; e_id,cus,ta,gd1,gd2 Joe,A,1,85,90 Joe,A,1,80,99 Joe,A,2,50,70 Joe,A,2,60,65 Joe,A,2,87,89 Joe,B,1,82,92 Joe,B,3,30,51 Rob,A,1,64,77 Rob,B,2,20,32 DATA close $resFile; my $dbh = DBI->connect("DBI:CSV:") or die $DBI::errstr; my $sth = $dbh->prepare("SELECT * FROM $tableName"); $sth->execute(); my $query = $dbh->selectall_arrayref( "select e_id, cus, ta, gd1, gd2 from $tableName order by e_id, cus +, ta", {Slice => {}}); my $gd1Total; my $gd2Total; my $currGroup; for my $row (@$query, {map {$_, 0} qw(e_id cus ta gd1 gd2)}) { my $group = join ',', @{$row}{qw(e_id cus ta)}; $currGroup = $group if !defined $currGroup; if ($group ne $currGroup) { print " Total $gd1Total, $gd2Total\n"; $gd1Total = $gd2Total = 0; $currGroup = $group; last if $group eq '0,0,0'; } print +(join ', ', @{$row}{qw(e_id cus ta gd1 gd2)}), "\n"; $gd1Total += $row->{gd1}; $gd2Total += $row->{gd2}; } exit;

        Prints:

        Joe, A, 1, 85, 90 Joe, A, 1, 80, 99 Total 165, 189 Joe, A, 2, 50, 70 Joe, A, 2, 60, 65 Joe, A, 2, 87, 89 Total 197, 224 Joe, B, 1, 82, 92 Total 82, 92 Joe, B, 3, 30, 51 Total 30, 51 Rob, A, 1, 64, 77 Total 64, 77 Rob, B, 2, 20, 32 Total 20, 32

        You'll have to fix the 'total' calculation to get the results you show however. My sample code simply sums the values.

        True laziness is hard work
Re: hoh or hoa data structure
by eff_i_g (Curate) on Jan 11, 2011 at 00:29 UTC
    I agree with Generoso: use SQL when possible.

    P.S. Here's your structure:
    use warnings; use strict; use Data::Dump qw(pp); my %hash; while (<DATA>) { chomp; next if /\A\s*\z/; my @pieces = split ',' => $_; for my $idx (3..4) { push @{ $hash {$pieces[0]} {$pieces[1]} {$pieces[2]} [$idx-3] }, $pieces[$idx]; } } pp %hash; __DATA__ Joe,A,1,85,90 Joe,A,1,80,99 Joe,A,2,50,70 Joe,A,2,60,65 Joe,A,2,87,89 Joe,B,1,82,92 Joe,B,3,30,51 Rob,A,1,64,77 Rob,B,2,20,32
    ( "Joe", { A => { 1 => [[85, 80], [90, 99]], 2 => [[50, 60, 87], [70, 65, 89] +] }, B => { 1 => [[82], [92]], 3 => [[30], [51]] }, }, "Rob", { A => { 1 => [[64], [77]] }, B => { 2 => [[20], [32]] } }, )
Re: hoh or hoa data structure
by ahmad (Hermit) on Jan 11, 2011 at 00:38 UTC

    If your only reason is finding the sum of the 4th & 5th column ... then a simple sql query might do it for you.

    Here's an example:

    select e_id,cus, sum(gd1) as g1, sum(gd2) as g2 from test group by e_i +d,cus,ta;

      SELECT null, e_id, Cus, ta, gd1, gd2 FROM tabla1 t union all select 'Total', e_id, Cus, ta, sum(gd1), SUM(gd2) from tabla1 group by + e_id,Cus, ta order by e_id,Cus, ta;

        Thanks to all for the different answers. It works perfect.