Couldn't have said it better :)

For the memory problem, a simple solution might be to have two classes.

my $buffer = Buffer->new();     # Creates a buffer and blessed SvUV.

my $shared = $buffer->share();  # Creates a blessed SvUV.
async {
   $shared->set(12345);
}->join();
undef $shared;  # Does nothing.

say $buffer->get();
undef $buffer;  # Frees the buffer.
[download]