Could it be that when you use this type of session that it caches the subroutine?

No, that merely creates references to the subroutines. You can update the subroutine at runtime, and as long as the reference points to the right thing, it will be fine. [If there's a way to do this, I don't know it. Everything I've tried creates a new coderef, which means any old references point to the old sub.]

Your problem is that when you undefine the sub, you are breaking the reference you previously created. A workaround to this could be:

POE::Session->new
    ( _start => \&bot_start,
        irc_001    => \&on_connect,
        irc_public => sub { on_public(@_) },
        irc_notice => \&on_notice,
        irc_msg    => \&on_msg,
    );
[download]

Update: fixed a few errors.

Another update: simonm's method may be preferable, as it uses the &subroutine calling variety to pass @_ implicitly. Another way would be goto &subroutine which replaces the current entry in the call stack.