There's nothing fundamentally wrong with your approach. The main difference is that using a separate hash for each property means you get "automatic spellchecking" of properties:

use strict;
my %prop;

sub method {
  my ($self) = @_;
  $prXp{id($self)} = 1;  # throws an error: $prXp is not declared
}
[download]

And using separate hashes means less typing.

One other thing: using refaddr is not safe when using fork() (on windows) or threads (on any system) since the refaddr changes over a CLONE. Object::InsideOut seems to deal with this issue.