in reply to How can I make a hash use less memory
why bother trying to store "a huge amount of data" in memory, when all you are doing is processing it linearly?
Sort your huge data file (by the key) first, and then process it with something like this...
#!/usr/local/bin/perl use warnings; use strict; my ($key, $first, $second) = split ' ', <>; while (<>) { my @line = split; if ($key eq $line[0]) { $first += $line[1]; $second += $line[2]; } else { print "$key $first $second\n"; ($key, $first, $second) = @line; } } print "$key $first $second\n";
(something like "sort your/big/ass/data/file.txt | monk.pl")
|
|---|