It looks like kyle already posted a good example of what you want, but I thought you might like a bit of explanation as to why your existing solution didn't work. Your code was:

$_[KERNEL]->alarm( $self->method => time() + 5 );

This will call $self->method right away, and pass its return value to $_[KERNEL]->alarm(). Then, later, POE will try to trigger an event named whatever your method returned.

In contrast, the code kyle posted wraps the call to $self->method in an event handler that gets called when the alarm triggers. You need the wrapping to avoid calling the method too soon.