We can't rely on a plan because we can't force authors what to do. Also, if the failure occurs after all tests have run, the plan is still correct. However, you've put me on th right track. I think there's a bug in Test::Builder and not with $SIG{__DIE__}.

  $ perl -le '$SIG{__DIE__} = sub { die }; foo()'; echo $?
  Died at -e line 1.
  255
[download]