An END { ... } block can be put into a closure as well:

{
    my $fn;
    do { $fn = tmpnam() }
        until open ...;
    END {
        unlink $fn or warn "Couldn't remove '$fn': $!";
    };
};
[download]

The die is especially useless indeed - just warning is far saner, as the program is terminating anyway and other cleanup likely doesn't rely on the removal of a temporary file ...