in reply to references troubles

This should work also -- the "$$href" part means that you are dereferencing $href and looking to identify a scalar value somewhere in the referenced structure; this way, you get to avoid using "->" altogether:
while(($key, $value) = each(%$href)) { $$href{$key}[1][0][0] = 1; ... print "$key/1/0/0 = $$href{$key}[1][0][0]\n"; }
Note that if you wanted to refer to an entire array that is referenced in the structure, you need to use extra curly brackets around the expression that follows the @ sigil:
my @toparray = @{$$href{$key}}; my @midarray = @{$$href{$key}[1]}; my @endarray = @{$$href{$key}[1][0]};