overload can do this, tie can't (though it's possible to also tie the array whose reference you want your object to pretend to contain).

package Foo::Bar;
use strict;
use warnings;

my @aoa;
my $cnt;
sub new { bless \(my $obj_index = $cnt++) }
sub DESTROY { undef $aoa[${$_[0]}] }
use overload '@{}' => sub { $aoa[${$_[0]}] ||= [] }, fallback => 1;
1;
[download]