The way to do this, without relying on the fact that the $p is a hashref, is to pass a closure as the handlers, and have an object that you created saved in the closure. This is how PerlSAX is implemented.

Witness:

my $handler = bless {}, "MyHandler";
my $p = XML::Parser->new(Handlers => {
   Start => sub { $handler->handle_start(@_) } 
});

package MyHandler;

sub handle_start {
  my ($handler, $p, $element, %attribs) = @_;
  ...
}