This behaviour is called "autovivification", and is a feature, not a bug of Perl. The background is that you treat $a[3] as a reference, and then perl thinks "if it should be a reference, let's make it one" and humbly turns an undef into a hash reference.

If you don't want autovivification to happen, you can use modules such as Data::Diver to access elements of nested data structures.

Since this behaviour confuses some people, Perl 6 autovivifies only if in lvalue context (ie if stuff is being assigned to it).