I would keep a list of files to delete, and delete them in an END block:

push @CLEANUP, $ARGV
    if $oldargv ne $ARGV;
END { unlink $_ for @CLEANUP };
[download]

... of course, that somewhat circumvents the idea of backup files, as that END block will still wipe all files ...