Your three-deep loop could be written,

my @array = (undef) x $size[0];
$_ = [ map { [(0) x $size[2]] } (undef) x $size[1] ] for @array;
[download]

Inside the map block is what happens in your deepest loop level. That is a reference to an array of $size[2] zeros. The list map acts on is bogus. Any list of $size[1] items would do there because the data in that list is never touched or remembered. It's only there to make $size[1] different copies of the inner array. A reference is taken to the whole map expression to make the top level data in @array.

Setting 3d array elements is easier than you think. Just say,

$array[$foo][$bar][$baz] = $quux
    if $foo >= -$size[0] and $foo < $size[0]
        and $bar >= -$size[1] and $bar < $size[1]
        and $baz >= -$size[2] and $baz < $size[2];
[download]

If you don't care about checking index range, all that logic can go.

blokhead's lvalue sub for fetch and store is an elegant approach.