I was seriously confused about this. After a bit of debugging I figured it out though. Here is a simple example that shows what is going on:

my $test = sub {bless {}};
$test->();
$test->();
$test->();

sub DESTROY {
  print "Calling DESTROY\n";
  die "But you won't see this message\n";
}
[download]

See it?

DESTROY is called in an eval. Since my code was only putting out output within a confess, that output was trapped. So I saw nothing, the script kept on going, and I thought that DESTROY wasn't being called. When in fact it is...

Mystery solved. :-)