If I remember right, Win32 doesn't let you unlink a file while anyone still has a handle open on it

No, that is just a type of sharing that you can specify. The C RTL defaults to opening files sharing both read and write access but not sharing rename/delete access.

If you want to delete a file while you have it open in Perl, you can use Win32API::File to open the file and specify that you want to allow FILE_SHARE_DELETE.