use List::Util qw(sum max min);

use strict;

my $keysfile = 'keys.txt';
my $valsfile = 'vals.txt';

open(KEYS, $keysfile) or die "Can't open $keysfile: $!";
open(VALS, $valsfile) or die "Can't open $valsfile: $!";

my %data;

while (!eof(KEYS) && !eof(VALS)) {
    chomp(my $key = <KEYS>);
    chomp(my $val = <VALS>);
    
    push @{$data{$key}}, $val;
}

warn "Premature end of $keysfile\n" if !eof(VALS);
warn "Premature end of $valsfile\n" if !eof(KEYS);

foreach my $name (sort keys %data) {
    my $values = $data{$name};

    my $count   = @$values;
    my $total   = sum(@$values);
    my $average = $total / $count;
    my $max     = max(@$values);
    my $min     = min(@$values);
    
    print "$name => total=$total, average=$average, max=$max, min=$min
+\n";
}
[download]

Thanks, that is a very nice solution. I didn't think of the idea of storing lists under each key then processing them afterward, and didn't know about the eof test.

And pretty code too.

Thanks again.

use strict;

use vars qw (%data);

open(KEYS, "keys.txt") or die "error: keys.txt $!";
open(VALS, "values.txt") or die "error: values.txt $!";

while (my $key = readline(*KEYS)) {
    my $val = readline(*VALS);
    chomp $key;
    chomp $val;

    print "$key = $val\n";
    unless (exists $data{$key}) {
        $data{$key}{min} = $data{$key}{max} = $data{$key}{total} = $va
+l;
        $data{$key}{count} = 1;
    } else {
        $data{$key}{min} = $val if $data{$key}{min} > $val;
        $data{$key}{max} = $val if $data{$key}{max} < $val;
        $data{$key}{total} += $val;
        $data{$key}{count}++;
    }
}

use Data::Dumper;
print Dumper \%data;
[download]

Thanks. Nice looking code.

Why readline(*KEYS) instead of <KEYS> ?

The readline function is what is used "behind the scenes" to implement the <> operator, and takes a typeglob, hence *KEYS instead of <KEYS>;

Keeping track of the min, max, total and count (for average value) may be faster than using the List utilities described earlier, or at least should take less memory, since you only need to keep track of a few values for each key. Although the utilities in question are extremely fast, you might need an array of thousands of values for each key, growing in size as you process the files.

I would suggest that your method as documented is both clear and concise. Remember that you are developing code that may need maintenance in months or years by you or someone else. You could play all sorts of obfuscation tricks that look more efficient, but the result will be inflexible code that is hard to read and maintain.

"...many millions of lines ... (big) files..."

It may be worth considering a disk based hash like DBM::Deep.

my ( $key,$value );

# or...

my ( %tips_by_name, %av_by_name, %max_by_name, %min_by_name );
[download]

my $key   = '';
my $value = '';
my %tips_by_name = '';
my %av_by_name = '';
my %max_by_name = '';
my %min_by_name = '';
[download]

Why on earth would you do that?

The scalars $keys and $value don't need initialization, but initializing to an empty string at least makes sense. Initializing a hash as in

my %tips_by_name = '';
[download]

is, pardon the bluntness, plain nonsense. Under warnings, Perl warns about it.

Anno

Initializing a variable in dmorgo's code would also be trivial. Your suggestion forces the programmer to do a lot of extra typing up front in order to maybe save a small amount of typing later, which seems like a poor tradeoff.

I would also argue that dmorgo's code has a clarity advantage in that it is immediately obvious which variables don't have default values and which do.

sub paste {
    my @fh = @_;
    return sub {
        chomp( my @line = map { scalar(<$_>) } @fh );
        return join ' ', @line;
    }
}
[download]

open my $keys, "keys.txt" or die "couldn't open keys.txt";
open my $vals, "vals.txt" or die "couldn't open vals.txt";
my $combined = paste($keys,$vals);
print $combined->(), "\n" for 0..4;
[download]

Joe 5
Bob 3
Sally 7
Bob 1
Fred 1
[download]

sub NEXTLINE { $_[0]->() }
print NEXTLINE($combined), "\n" for 0..4;
[download]