In addition to the (++) previous answer, Windows never lets user delete an open file or directory. If the file you want to delete is open, you'll have to terminate the application which holds it open (or make it close the file somehow else).
Sorry if my advice was wrong.
Comment on Re: Can't remove directory-Permission denied
Not completely true. Windows applications are free to open files in a manner that specifies that deleting is allowed. It is just a matter of which FILE_SHARE_* constants are specified. I find it unfortunate that the Unix-like interfaces for opening files on Windows specify that renaming and deleting is not to be allowed. I'd prefer that Unix-like calls result in more-Unix-like behavior.
Win32API::File documents the FILE_SHARE_* constants and allows one to make use of them from Perl. Sadly, I neglected to mention in those documents that FILE_SHARE_DELETE also allows the file to be renamed while it is still open.