Upgrade to a newer version of CGI::Carp that knows to check $^S. Alternately you could use local( $SIG{__DIE__} ); which I think will work to temporarilly disable CGI::Carp from catching die calls.

Update: Wow, I could've swore I read "eval block" in the original question. See tilly's answer.