Heres my slightly different take on it. When I first saw this problem, I had an immediate desire to (ab)Data::Dumper's dumper sub to avoid having to do my own serialization/string conversion. In short I wanted to do Dumper($ha) eq Dumper($hb). Unfortunately to do so I had to write a brief routine that sorts the AoAs.

First I iterate through the child arrays and sort each one, then I sort the parent array. After that it's a simple matter of doing Dumper($ha) eq Dumper($hb) which is what I wanted in the first place =].

use strict;
my $ha = [[1,2],
         [5,5,5],
         [1,2],
         [3,4]];

my $hb = [[4,3],
         [2,1],
         [1,2],
         [5,5,5]];

sub test
{
    my ($ha, $hb) = @_;
    
    my ($tha,$thb) =  ([],[]);
    
    for( @{ $ha } )
    {
        push @{$tha},[ sort @{$_} ];
    }
    
    $tha = [ sort { $a->[0] <=> $b->[0] } @{$tha} ];
    
    for( @{ $hb } )
    {
        push @{$thb},[ sort @{$_} ];
    }
    
    $thb = [ sort { $a->[0] <=> $b->[0] } @{$thb} ];
    
    use Data::Dumper;
    
    return Dumper($tha) eq Dumper($thb);
}

print test($ha,$hb);
[download]

Note that it only handles AoA, and theres some ugly repeated code (the two for loops that sort the temporary array refs ), and basically no error checking. But it does seem to work =].