|
Re^4: locking over the network (rename)
by rovf (Priest) on Feb 01, 2011 at 10:11 UTC
|
I didn't consider rename, because rename says:
... it will usually not work across file system boundaries, even though the system mv command sometimes compensates for this.
As for atomicity, I wonder how Perl could guarantee that, without assumption of the file system being used. Unfortunately I don't know enough about Samba, CIFS and NFS, to know what they offer for atomic rename. May I ask you where you got the information that rename is atomic?
--
Ronald Fischer <ynnor@mm.st>
| [reply]
[d/l]
[select] |
|
|
NFS is notable for not supporting atomic rename. I'm sure that SMB/CIFS supports atomic rename, even over the network, but the Wikipedia pages for NFS and SMB are silent on the features of the filesystems/protocols.
| [reply] |
|
|
NFS is notable for not supporting atomic rename.
Where do you get this? Lots of versions of rename,2 say
If newpath already exists it will be atomically replaced (subject to a few conditions; see ERRORS below), so that there is no point at which another process attempting to access newpath will find it missing.
Which covers exactly the race that was discussed. And the "ERRORS" section mentions nothing of NFS. Yet NFS is specifically covered in the "BUGS" section:
On NFS file systems, you can not assume that if the operation failed the file was not renamed. If the server does the rename operation and then crashes, the retransmitted RPC which will be processed when the server is up again causes a failure. The application is expected to deal with this. See link(2) for a similar problem.
Which says nothing about the rename not happening atomically. It just means that ENOENT might result not because somebody else (re)moved the "old" name before you could call rename(2) but also because you renamed it before retrying the operation due to remote server failure.
| [reply] |
|
|
I'm sure that SMB/CIFS supports atomic rename
I'm quite optimistic that it does, but the question is whether this is used "automatically" by Perl or not. My guess is that Perl just calls the 'rename' function of the C library, when it was compiled. I don't know how to verify this because I am using the ActiveState distribution of Perl 5.8.8, and don't know where to get the sources (ActiveState doesn't seem to have them anymore). But even if I have this information, I would need to know which version of which C compiler was used to create this distribution, and whether the rename in the compiler's library works in the expected way.
That's why I hoped that the Perl documentation would give some "guarantees" about this issue, but even perlport doesn't say anything about atomicity.
You mention that NFS does not support atomic rename. I didn't know this before - thank you for telling me -, and since access to the filesystem via NFS is one possibility for our case, I think I should better drop the idea of replacing my original solution (using flock) by the more elegant 'rename' solution. :-(
--
Ronald Fischer <ynnor@mm.st>
| [reply]
[d/l] |
|
|
|
|
|
|
|
|
|
|
|
|
Note the "file system boundaries". What that is referring to is trying to rename a file from one drive/partition to another, not watching the rename happen on a remote system.
IE: You can't rename a file from your hard disk to a floppy disk. Just doesn't make sense. Any such rename/mv operation would be done via a (very slow) copy, then a delete.
On the other hand, if you're renaming a file on the same partition, or in this case the same directory even, then all you need to do is change the name; the physical location of the file on the media doesn't need to change, and the file contents won't be partially written at any point during that operation. You simply won't see the file until the name begins matching the string you're looking for.
| [reply] |
|
|
You are writing the file to one file system and then "renaming" it to a different file system? Why? That's rather silly. Renaming to a different file system requires that the contents be copied, which means races leading to partial content. If you need to do that, then I'd copy the file to the new file system under a temporary name (even if you use /bin/mv to do the copying) and then atomically rename from the temporary name (which the receiver should know to ignore) to the real name (which might involve two different directories on the final file system).
You haven't even bothered to read any Unix man pages for rename? Did you want an engraved invitation? They even answer your question about NFS.
Note that Perl's rename is atomic on Win32 including across Samba. No, I don't have links that say this ready to hand to you. You might have to do some research.
| [reply] |
|
|
| [reply]
[d/l] |
|
|
|
|
| [reply] |