Others have adequately explained why $hash{$key} won't work (since this accesses %hash, not a reference in $hash). The reason why $array[$index]{$key} works when $array[$index] is a reference is because Perl makes it work for you. Perl tries to make accessing complex data structures easy for you, and since $array[$index] can't be anything but a reference to a hash, there's no harm or ambiguity in treating $array[$index}{$key} like $array[$index]->{$key}. You can reference any level of data this way.

Please read perlref, perlreftut, perldsc and perllol for information about references and complex data structures with them.