Hash keys must be strings, although Tie::RefHash uses magic to provide what looks to be a hash that accepts non-strings as keys.

I can get access to the arrays again if I access a specific value with a specific key

Yes and no.

This works:

my $k = [qw( a b )];
$h{$k} = $v;
$v = $h{$k};
[download]

This doesn't (except occasionally by fluke):

$h{ [qw( a b )] } = $v;
$v = $h{ [qw( a b )] };
[download]

You're using the address of the array as the key, so if you create a similar array at a different address, it won't be the same key.