http://qs1969.pair.com?node_id=11142758

Special_K has asked for the wisdom of the Perl Monks concerning the following question:

I have the following code intended to sort a hash of hashes by value:

#!/usr/bin/perl -w use strict; my %hoh_test; $hoh_test{foo1}{bar} = -0.12697; $hoh_test{foo1}{baz} = -0.000398154; $hoh_test{foo2}{bar} = -4.0183e-05; $hoh_test{foo2}{baz} = 0; $hoh_test{foo3}{bar} = 9.966003977e-06; $hoh_test{foo3}{baz} = 0.0001939; # sort descending by value foreach my $foo (keys(%hoh_test)) { foreach my $ba (sort {$hoh_test{$foo}{$b} <=> $hoh_test{$foo}{$a}} + keys(%{$hoh_test{$foo}})) { printf("foo: $foo, ba: $ba, value: %s\n", $hoh_test{$foo}{$ba} +); } }

Here is the expected output:

foo: foo3, ba: baz, value: 0.0001939 foo: foo3, ba: bar, value: 9.966003977e-06 foo: foo2, ba: baz, value: 0 foo: foo2, ba: bar, value: -4.0183e-05 foo: foo1, ba: baz, value: -0.000398154 foo: foo1, ba: bar, value: -0.12697

The actual output varies most times the program is run, so clearly my code doesn't work as intended. My two questions are:

1. How can I fix this to sort by values?
2. What exactly is it even doing now? Given that the output changes with most runs it doesn't appear to be sorting by anything.

Replies are listed 'Best First'.
Re: help sorting hash of hashes by value
by graff (Chancellor) on Apr 07, 2022 at 01:10 UTC
    1. You can fix it by creating a separate hash of arrays, with the sortable values as keys, and the 2-layer (original) key combinations as values.

    2. As originally posted, your code is first looping over the "foo" keys (in whatever random order perl is using to return that set of hash keys, which differs from one run to the next), and then only the "ba" keys are being sorted according to the values tied to each key.

    Other monks can probably provide a more elegant solution, but here's the first thing that comes to mind, sticking as close as possible to the OP code:

    #!/usr/bin/perl -w use strict; my %hoh_test; $hoh_test{foo1}{bar} = -0.12697; $hoh_test{foo1}{baz} = -0.000398154; $hoh_test{foo2}{bar} = -4.0183e-05; $hoh_test{foo2}{baz} = 0; $hoh_test{foo3}{bar} = 9.966003977e-06; $hoh_test{foo3}{baz} = 0.0001939; # sort descending by value my %sorter; foreach my $foo (keys(%hoh_test)) { foreach my $ba (sort {$hoh_test{$foo}{$b} <=> $hoh_test{$foo}{$a}} + keys(%{$hoh_test{$foo}})) { push @{$sorter{$hoh_test{$foo}{$ba}}}, [ $foo, $ba ]; } } foreach my $val (sort {$a<=>$b} keys %sorter) { foreach my $keys ( @{$sorter{$val}} ) { print "foo: $$keys[0], ba: $$keys[1], value: $hoh_test{$$keys[ +0]}{$$keys[1]}\n"; } }
    The output I get from that is consistently:
    foo: foo1, ba: bar, value: -0.12697 foo: foo1, ba: baz, value: -0.000398154 foo: foo2, ba: bar, value: -4.0183e-05 foo: foo2, ba: baz, value: 0 foo: foo3, ba: bar, value: 9.966003977e-06 foo: foo3, ba: baz, value: 0.0001939
    UPDATE: OOPS!! I forgot that you wanted the output sorted in descending order -- so for that, you just need to change {$a<=>$b} to be {$b<=>$a} in the second outer for loop.

    ANOTHER UPDATE: (Silly me...) I realized that I was sticking too close to the OP code... no sorting is required in the first (nested) for loop -- here's the code again, without the unnecessary sort, and with sort direction corrected:

    #!/usr/bin/perl -w use strict; my %hoh_test; $hoh_test{foo1}{bar} = -0.12697; $hoh_test{foo1}{baz} = -0.000398154; $hoh_test{foo2}{bar} = -4.0183e-05; $hoh_test{foo2}{baz} = 0; $hoh_test{foo3}{bar} = 9.966003977e-06; $hoh_test{foo3}{baz} = 0.0001939; # sort descending by value my %sorter; foreach my $foo (keys(%hoh_test)) { foreach my $ba (keys(%{$hoh_test{$foo}})) { push @{$sorter{$hoh_test{$foo}{$ba}}}, [ $foo, $ba ]; } } foreach my $val (sort {$b<=>$a} keys %sorter) { foreach my $keys ( @{$sorter{$val}} ) { print "foo: $$keys[0], ba: $$keys[1], value: $hoh_test{$$keys[ +0]}{$$keys[1]}\n"; } }
Re: help sorting hash of hashes by value
by kcott (Archbishop) on Apr 07, 2022 at 08:12 UTC

    G'day Special_K,

    Your basic problem is not taking into account the fact that hashes are unordered. You'll get foo1, foo2 & foo3 returned in a random order when you call keys %hoh_test; and, for each of those, you'll get bar & baz returned in a random order when you call keys %{$hoh_test{$foo}}.

    Here's my take on a solution:

    #!/usr/bin/env perl use strict; use warnings; my %hoh_test = ( foo1 => { bar => -0.12697, baz => -0.000398154 }, foo2 => { bar => -4.0183e-05, baz => 0 }, foo3 => { bar => 9.966003977e-06, baz => 0.0001939 }, ); my @hoh_data; for my $k0 (keys %hoh_test) { for my $k1 (keys %{$hoh_test{$k0}}) { push @hoh_data, [$k0, $k1, $hoh_test{$k0}{$k1}]; } } my $fmt = "foo: %s, ba: %s, value: %s\n"; printf $fmt, @$_ for sort { $b->[2] <=> $a->[2] } @hoh_data;

    This produces the same output on each run:

    foo: foo3, ba: baz, value: 0.0001939 foo: foo3, ba: bar, value: 9.966003977e-06 foo: foo2, ba: baz, value: 0 foo: foo2, ba: bar, value: -4.0183e-05 foo: foo1, ba: baz, value: -0.000398154 foo: foo1, ba: bar, value: -0.12697

    Just to exemplify the unordered nature of hashes, here's what @hoh_data looked like, on two separate runs, before passing to sort { $b->[2] <=> $a->[2] }:

    ( ["foo1", "bar", -0.12697], ["foo1", "baz", -0.000398154], ["foo2", "baz", 0], ["foo2", "bar", -4.0183e-05], ["foo3", "baz", 0.0001939], ["foo3", "bar", 9.966003977e-06], )
    ( ["foo1", "baz", -0.000398154], ["foo1", "bar", -0.12697], ["foo3", "baz", 0.0001939], ["foo3", "bar", 9.966003977e-06], ["foo2", "bar", -4.0183e-05], ["foo2", "baz", 0], )

    Update: Well, I made a real mess of keys %{hoh_test($foo}}. Now fixed: keys %{$hoh_test{$foo}}.

    — Ken

Re: help sorting hash of hashes by value
by cavac (Parson) on Apr 07, 2022 at 07:07 UTC

    If your amount of data is sort of limited and fits into RAM, the answer by by graff is the way to go. If the data gets bigger, or more complex, or you also start needing more complex sorting/filtering, or you start running into performance problems, you might want to look at using a proper SQL database. For example, PostgreSQL is free, and Perl has excellent support through DBI/DBD::Pg.

    Modern databases are super optimized to deal with exactly those kinds of problems. Plus, with PostgreSQL you get the benefit that you can implement server side smarts in Perl (among other programming languages).

    perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'
Re: help sorting hash of hashes by value
by atcroft (Abbot) on Apr 08, 2022 at 01:18 UTC

    If I read your code correctly, only the inner loop on $ba is sorting on values. Hash keys are not returned in a specific order, so your $foo keys may be all over the place. But your expected output really looks to be sorting on key $foo, then on $ba, so I'm not sure "sorting on values" is you are asking. If it is, I would expect the following code snippet to generate your expected output:

    use strict; use warnings; my %hoh_test = ( foo1 => { bar => -0.12697, baz => -0.000398154, }, foo2 => { bar => -4.0183e-05, baz => 0, }, foo3 => { bar => 9.966003977e-06, baz => 0.0001939, }, ); # Sort on $foo decending foreach my $foo ( sort { $b cmp $a } keys %hoh_test ) { # Sort on values decending foreach my $ba ( sort { $hoh_test{$foo}{$b} <=> $hoh_test{$foo}{$a} } keys %{$hoh_test{$foo}} ) { printf "foo: %s, ba: %s, value: %s\n", $foo, $ba, $hoh_test{$foo}{$ba}; } }

    The output matches your expected output:

    If instead you are sorting on $foo decending then on $ba decending, the code would instead be:

    use strict; use warnings; my %hoh_test = ( foo1 => { bar => -0.12697, baz => -0.000398154, }, foo2 => { bar => -4.0183e-05, baz => 0, }, foo3 => { bar => 9.966003977e-06, baz => 0.0001939, }, ); # Sort on $foo decending foreach my $foo ( sort { $b cmp $a } keys %hoh_test ) { # Sort on $ba decending foreach my $ba ( sort { $b cmp $a } keys %{$hoh_test{$foo}} ) { printf "foo: %s, ba: %s, value: %s\n", $foo, $ba, $hoh_test{$foo}{$ba}; } }

    This output also matches your expected output:

    If you really want your hash output sorted by values, there may be other ways but the simplest that comes to my mind is to push the order into a temporary structure you can actually sort:

    use strict; use warnings; use utf8; my %hoh_test = ( foo1 => { bar => -0.12697, baz => -0.000398154, }, foo2 => { bar => -4.0183e-05, baz => 0, }, foo3 => { bar => 9.966003977e-06, baz => 0.0001939, }, ); # Sort on values by pushing into an intermediary structure my @ordering; foreach my $foo ( keys %hoh_test ) { foreach my $ba ( keys %{ $hoh_test{$foo} } ) { push @ordering, { foo => $foo, ba => $ba, v => $hoh_test{$foo} +{$ba}, }; } } # Sort intermediary structure on value (v) from above. foreach my $i ( sort { $ordering[$b]{v} <=> $ordering[$a]{v} } 0 .. $# +ordering ) { printf "foo: %s, ba: %s, value: %s\n", $ordering[$i]{foo}, $ordering[$i]{ba}, $ordering[$i]{v}; }

    This also creates your expected output:

    Hope that helps.

Re: help sorting hash of hashes by value
by johngg (Canon) on Apr 08, 2022 at 15:40 UTC

    I first approached this by forming a "flattened" intermediate hash where the first and second level keys were joined with a NUL character. The keys of the new hash could then be sorted by the associated descending numeric values then used for the printf statement. This code

    use strict; use warnings; my %hoh_test = ( foo1 => { bar => -0.12697, baz => -0.000398154, }, foo2 => { bar => -4.0183e-05, baz => 0, }, foo3 => { bar => 9.966003977e-06, baz => 0.0001939, }, ); my %flatter = map { my $k1 = $_; map { join( qq{\0}, $k1, $_ ), $hoh_test{ $k1 }->{ $_ } } keys %{ $hoh_test{ $k1 } }; } keys %hoh_test; my @sortedKeys = sort { $flatter{ $b } <=> $flatter{ $a } } keys %flatter; printf qq{foo: %s, ba: %s, value: %s\n}, split( m{\0}, $_ ), $flatter{ $_ } for @sortedKeys;

    produces

    foo: foo3, ba: baz, value: 0.0001939 foo: foo3, ba: bar, value: 9.966003977e-06 foo: foo2, ba: baz, value: 0 foo: foo2, ba: bar, value: -4.0183e-05 foo: foo1, ba: baz, value: -0.000398154 foo: foo1, ba: bar, value: -0.12697

    This seemed a bit long-winded so I thought it might be possible to still use combined keys but mapping out array refs to perform an ST all in one fell swoop. This code

    use strict; use warnings; my %hoh_test = ( foo1 => { bar => -0.12697, baz => -0.000398154, }, foo2 => { bar => -4.0183e-05, baz => 0, }, foo3 => { bar => 9.966003977e-06, baz => 0.0001939, }, ); printf qq{foo: %s, ba: %s, value: %s\n}, @{ $_ } for map { [ split( m{\0}, $_->[ 0 ] ), $_->[ 1 ] ] } sort { $b->[ 1 ] <=> $a->[ 1 ] } map { my $k1 = $_; map { [ join( qq{\0}, $k1, $_ ), $hoh_test{ $k1 }->{ $_ } ] } keys %{ $hoh_test{ $k1 } }; } keys %hoh_test;

    produces identical output. I hope this is of interest.

    Update: Corrected typo, s/tha/the/

    Cheers,

    JohnGG