What uploaded file are you trying to delete? CGI.pm will automatically delete the temporary file it writes the upload into. It usually deletes the file when the filehandle is destroyed. The $PRIVATE_TEMPFILES variable causes it to unlink the file before passing the open handle(I don't think it will do this on Win32).

How are you getting the file names to delete? Are you reading the directory with opendir/readdir or glob? Does it check that the file exists before trying to delete it?