in reply to Re^4: locking over the network (rename)
in thread locking over the network

NFS is notable for not supporting atomic rename. I'm sure that SMB/CIFS supports atomic rename, even over the network, but the Wikipedia pages for NFS and SMB are silent on the features of the filesystems/protocols.

  • Comment on Re^5: locking over the network (rename)

Replies are listed 'Best First'.
Re^6: locking over the network (NFS)
by tye (Sage) on Feb 01, 2011 at 18:34 UTC
    NFS is notable for not supporting atomic rename.

    Where do you get this? Lots of versions of rename,2 say

    If newpath already exists it will be atomically replaced (subject to a few conditions; see ERRORS below), so that there is no point at which another process attempting to access newpath will find it missing.

    Which covers exactly the race that was discussed. And the "ERRORS" section mentions nothing of NFS. Yet NFS is specifically covered in the "BUGS" section:

    On NFS file systems, you can not assume that if the operation failed the file was not renamed. If the server does the rename operation and then crashes, the retransmitted RPC which will be processed when the server is up again causes a failure. The application is expected to deal with this. See link(2) for a similar problem.

    Which says nothing about the rename not happening atomically. It just means that ENOENT might result not because somebody else (re)moved the "old" name before you could call rename(2) but also because you renamed it before retrying the operation due to remote server failure.

    - tye        

Re^6: locking over the network (rename)
by rovf (Priest) on Feb 01, 2011 at 16:00 UTC
    I'm sure that SMB/CIFS supports atomic rename
    I'm quite optimistic that it does, but the question is whether this is used "automatically" by Perl or not. My guess is that Perl just calls the 'rename' function of the C library, when it was compiled. I don't know how to verify this because I am using the ActiveState distribution of Perl 5.8.8, and don't know where to get the sources (ActiveState doesn't seem to have them anymore). But even if I have this information, I would need to know which version of which C compiler was used to create this distribution, and whether the rename in the compiler's library works in the expected way.

    That's why I hoped that the Perl documentation would give some "guarantees" about this issue, but even perlport doesn't say anything about atomicity. You mention that NFS does not support atomic rename. I didn't know this before - thank you for telling me -, and since access to the filesystem via NFS is one possibility for our case, I think I should better drop the idea of replacing my original solution (using flock) by the more elegant 'rename' solution. :-(

    -- 
    Ronald Fischer <ynnor@mm.st>

      Windows Perl gets built from the same source as Other Perl, so you don't need anything from ActiveState.

      Perl calls just the underlying C library, and commonly, at least on POSIX 1 implementations, the C library is a thin wrapper around the rename(3) system call. On Windows, the call to rename is in win32.c.

        On Windows, the call to rename is in win32.c.

        Thank you for this link! From what I can see, this shows that rename indeed is *not* atomic on Windows!

        -- 
        Ronald Fischer <ynnor@mm.st>
      That's why I hoped that the Perl documentation would give some "guarantees" about this issue

      Really? After saying this:

      I don't know how to verify this because I am using the ActiveState ... and don't know where to get the sources ... I would need to know which version of which C compiler was used to create this distribution, and whether the rename in the compiler's library works in the expected way.

      So you hoped Perl docs would say "We can guarantee that no provider of Perl will ever change the source code nor use a C library such that ..." ?

      - tye        

        So you hoped Perl docs would say "We can guarantee that no provider of Perl will ever change the source code nor use a C library such that ..." ?
        For any feature of an implementation of a programming language - and Perl's rename is not an exception -, the implementation can either guarantee a certain behaviour, or let the behaviour be undefined. I don't know whether it is, in this case, possible to guarantee atomicity for rename. Reading through the postings in this thread, some contributors seem to be sure that I can rely on atomicity for the platforms I'm running Perl one, Corion doubted that this could be guaranteed with NFS (a point where rowdog disagrees), and so on. I don't have the knowledge to jugde about the conditions, under which I can rely on rename's atomicity, but, yes, I had hoped that IF such guarantees can be done, they would be documented at least in perlport. Of course, IF such guarantees can NOT be done, I don't expect the Perl docs to point this out (this would be too much to ask), but then it would be safer not to rely on this feature and stick with my cooperative solution regarded cooperative locking.

        Of course if a provider of a Perl distribution compiles Perl with a buggy compiler (say), then whatever guarantees we find in the Perl docs, they are not necessarily true anymore - the old "garbage-in, garbage-out" principle.

        -- 
        Ronald Fischer <ynnor@mm.st>