http://qs1969.pair.com?node_id=1154078


in reply to Merging partially duplicate lines

As poj suggested, in these modern days importing into a relational database like Microsoft Access or sqlite might provide an easy SQL solution to your problem. If you want to look at your files as a text database then the problem looks like it fits an old text database processing tool: Awk. Perl has an awkish/autosplit '-a' mode suggesting:

#!/usr/bin/perl -a use strict; use warnings; our %recs; my $k = join "\t", @F[0 .. 3]; if ($recs{$k}) { $recs{$k}->{key_count}++; $recs{$k}->{rec}->[$_] += $F[$_] for 4, 5; } else { # careful to copy with [ @F ] not \@F here $recs{$k} = {key_count => 1, rec => [ @F ]}; } END { $recs{$_}->{rec}->[4] /= $recs{$_}->{key_count} foreach keys %recs +; print join("\t",@{$recs{$_}->{rec}}), $/ foreach sort { $recs{$a}->{rec}->[ 0 ] cmp $recs{$b}->{rec}->[ 0 ] || $recs{$a}->{rec}->[ 1 ] <=> $recs{$b}->{rec}->[ 1 ] || $recs{$a}->{rec}->[ 2 ] cmp $recs{$b}->{rec}->[ 2 ] || $recs{$a}->{rec}->[ 3 ] cmp $recs{$b}->{rec}->[ 3 ] } keys %recs; }
Ron