eisenstein has asked for the wisdom of the Perl Monks concerning the following question:

Frequently, when processing rows returned by a DB query, we need to group sets of rows. We often want to display a single header for the group and/or calculate some aggregate values. Here is an abstract example.
@a = (['a',1],['a',2],['b',3],['b',4]); # "db read" my $old; my $total = 0; # init foreach my $row (@a) { if ($old && $old ne $row->[0]) { print "$old: $total\n"; # finish row block $total = 0; # init } $total += $row->[1]; # processrow block $old = $row->[0]; # check variable } print "$old: $total\n"; # finish row block
Bad things: But, I'd like not to: Anyone developed a clean solution for this? Any advice would be appreciated.

Replies are listed 'Best First'.
Re: How to loop over groups
by Juerd (Abbot) on Apr 02, 2002 at 08:07 UTC

    Anyone developed a clean solution for this? Any advice would be appreciated.

    If you use some SQL server, use its capabilities to do the work for you. I know MySQL can use:

    SELECT foo, SUM(bar) FROM xyzzy GROUP BY foo
    If you're not using SQL, or your SQL doesn't have this functionality, I'd sum first, and iterate later:
    my @data = ( [ 'a', 1 ], [ 'a', 2 ], [ 'b', 3 ], [ 'b', 4 ] ); my %totals; my @order; for (@data) { push @order, $_->[0] unless exists $totals{$_->[0]}; $totals{$_->[0]} += $_->[1]; } for (@order) { # Do something with $totals{$_} }
    If order is not important, leave out all @order stuff, and iterate over keys %totals;

    U28geW91IGNhbiBhbGwgcm90MTMgY
    W5kIHBhY2soKS4gQnV0IGRvIHlvdS
    ByZWNvZ25pc2UgQmFzZTY0IHdoZW4
    geW91IHNlZSBpdD8gIC0tIEp1ZXJk
    

      True. But it gets much worse when I'm printing out a header for the section, then the individual rows in that group, then the total. Thinking about it now, the trickiest part is knowing when to print out the next group's header. Is there any way to avoid using the "$old" temp variable to handle that?
      @a = (['a',1],['a',2],['b',3],['b',4]); # "db read" my $old; my $total = 0; # init foreach my $row (@a) { if ($old && $old ne $row->[0]) { print "$old: $total\n"; # finish row block $total = 0; # init print "Section $row->[0]\n"; } $total += $row->[1]; # processrow block print "$row->[1]\n"; $old = $row->[0]; # set temp variable } print "$old: $total\n"; # finish row block
        Sure, just rearrange your data.

        By building a hash which groups the rows together, your problem goes away.

        @a = (['a',1],['a',2],['b',3],['b',4]); # "db read" my %aByGroup; foreach(@a) { # you could just push $_->[1], but I'm guessing your # data is more complex, so I'm keeping the whole row push @{$aByGroup{$_->[0]}},$_; } my $old; foreach my $group (sort keys %aByGroup) { my $total = 0; # init print "Section $group\n"; foreach my $row(@{$aByGroup{$group}}) { $total += $row->[1]; # processrow block print "$row->[1]\n"; } print "$group: $total\n"; # finish row block }

        --
        Mike
Re: How to loop over groups
by dragonchild (Archbishop) on Apr 02, 2002 at 16:20 UTC
    init and finish sections are repeated! In the real code, these sections are both a dozen lines, and they constantly get out-of-sync: a major source of bugs.

    This calls for a function. It'll also make it easier to track your logic. :-)

    ------
    We are the carpenters and bricklayers of the Information Age.

    Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Re: How to loop over groups
by YuckFoo (Abbot) on Apr 02, 2002 at 18:43 UTC
    eisenstein,

    I always wanted a good solution to this myself. I've done it that way more times than I'd like to admit. Adding a look-ahead removes a lot of awkwardness of processing near the top of the loop and redundant code for the last record.

    Another advantage of this partial solution is you have the complete next record so you can easily check for changes in other fields.

    YuckFoo

    @a = (['a',1],['a',2],['b',3],['b',4]); $last = ['', '']; for $i (0..$#a) { $this = $a[$i]; $next = defined $a[$i+1] ? $a[$i+1] : $last; $total += $this->[1]; if ($this->[0] ne $next->[0]) { print "$this->[0]: $total\n"; $total = 0; } }