Expanding on what the anony monk said, your die handler can test if the die happened in an eval.

So, maybe the following:

# not tested
sub SigDieHandler
{
    unless ($^S) # skip if die happened in an eval
    {
        ...; # create and send email
    }
}
[download]

Not tested, YMMV.