I agree with ++dsheroh's assessment. Simplifying the data structure results in a simpler sort (i.e. easier to read and maintain).

Here's a quick-and-dirty example to show the simplified sort logic:

$ perl -E '
    my %x = (A=>{h=>2}, B=>{h=>1}, C=>{h=>3});
    say for sort { $x{$b}{h} <=> $x{$a}{h} } keys %x;
'
C
A
B
[download]