How can you dereference the value in $AoA1[0][1] when $AoA1[0][1] is obviously a scalar (with the value of 1 it seems) ?

You can only dereference a reference. '1' is not a reference

Please post more of your code. Language is always ambiguous, code (mostly) not.For example your statement "a reference is being assigned to @AoA2" is ambiguous at least and if taken at face value definitly not what you want. @AoA2= \$x; is equivalent to @AoA2= (); $AoA1[0]= \$x;. Makes no sense, right?

Also you might use Data::Dumper to get a clear picture of the data you are operating on (really Data::Dumper is the most often mentioned module on PerlMonks, and that's justified).