Your example works fine, and it will continue to work fine, but it loses one of the advantages of inside-out objects. With the usual way, if you mistype an attribute name somewhere (and you use strict), Perl will cough up an error message before it even runs anything.

sub get_x {
   return $xc00rd{refaddr $_[0]};  # oh noez!
}
[download]

The way you have it, Perl will just autovivify the attribute you hadn't created before.

sub get_x {
    # $coord{refaddr $_[0]}->{'X'} == 42
    return $coord{refaddr $_[0]}->{'ex'};  # but this is undef
}
[download]

This can lead to subtle and difficult bugs down the line, but only if you ever make mistakes while typing.

As an aside, consider using a framework to make inside-out classes easier (such as Object::InsideOut or Class::Std). These will create constructors, destructors, and accessors for you.