Given all of that, you probably should only read the info from the file that you need, then do your calculations. After that, discard the data you've gotten and go to the next group. (This will reduce your memory needs by at least 90%, given a typical bioinformatics dataset.)

Now, if you do that, you don't need to split your initial key into four keys. In fact, since all the data you're working with has to do with those, you don't need to use it as a key at all. So, all you have is your topic, your index, and your value - a standard 2-D data structure. Try working with that and see how far you get.

If your data isn't sorted, I would sort it first and save that to some file (or set of files). Then, use that as your data source for summations and the like.

------
We are the carpenters and bricklayers of the Information Age.

The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Here's what I would do.

First, get a list of all of the hash keys you want to sort. Since the hash is so deeply nested, it may be easiest to store the results in an array ref:

foreach my $k1 (keys %bighash) {
  foreach my $k2 (keys %{$bighash{$k1}}) {
    ...
       push(@keys,[$k1, $k2, ...]);
[download]

@keys = sort by_key_total @keys;

sub by_key_total
{
  $bighash{$a->[1]}{$a->[2]}{$a->[3]}{$a->[4]}[$a->[5]][$a->[6]]
     <=>
  $bighash{$b->[1]}{$b->[2]}{$b->[3]}{$b->[4]}[$b->[5]][$b->[6]]
}
[download]

Not true. Of course you can sort a hash. The random part is the order in which the keys get inserted, but if you do

sort(keys %hash)
[download]

you'll get the keys in alphabetical order. If you do

sort { $a <=> $b } keys %hash;
[download]

you'll get the keys in numerical order.

The only thing you can't get from the hash is the order in which things were added

Update: I can't type ... switched can get to can't get in the sentence above, sort of changes the meaning :)

Update: sauoq is correct and I'm completely wrong (not the first time) I wasn't thinking about the fact that sort is simply returning the keys in the order which you ask. The original remains the same. Apologies sweetblood.

Lobster Aliens Are attacking the world!

Of course you can sort a hash.

Well, no. You really can't. You can, as your examples show, generate a list of the sorted keys of a hash¹. You can do the same with the values. But the hash itself remains unsorted.

Contrast that with an array, a structure that you can sort:

perl -le '@a = qw(b c a); @a = sort @a; print "@a"'
[download]

The equivalent for a hash, something like %h = map { $_, $h{$_} } sort keys %h; does a lot of work for no good reason. (And, %h = sort %h wouldn't even keep the key/value pairs intact.) There is simply no way to sort a hash.

1. People are usually perfectly happy with this solution. My guess is that the monk to which you were responding was either being pedantic or was reciting something by rote that he recently learned.

-sauoq
"My two cents aren't worth a dime.";