Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Re: File lock demo

by davido (Cardinal)
on Apr 21, 2021 at 19:21 UTC ( [id://11131542] : note . print w/replies, xml ) Need Help??

in reply to File lock demo

In the context you are showing this is safe and effective. I just wanted to mention a couple of things relating to the open mode you are using, ">". In your use-case, the semaphore file is safe. But if someone were to come along and look at this as how to open an output file they could be exposing themselves to a nasty race condition. Also, a common idiom in using a semaphore lock file for highlander control is to write the lock-holder's PID into the lockfile. In both cases, the ">" open is risky.

Here's the scenario:

  1. First process opens an output file, obtains a lock, and writes something into the file.
  2. Second process opens the same output file, but fails to obtain a lock because the first process holds the lock. However, because the file was opened in ">" mode, the file's contents were truncated to zero. So the process that failed to obtain a lock modified the file that some other process had a lock on before attempting to get a lock.

It is sometimes useful to write a PID into a lockfile so that it is easier for an observer to know who holds the lock, or who is manipulating the file. This isn't strictly necessary; it's informative. I could, as an outsider to the software, come along and look, seeing that myscript.LOCK contains the pid 6327. I can look and see if that PID is still alive, how long it's been running, how much memory it's using, and so on. There are other ways to discover who holds a lock, but just looking at a PID written into a file is an easy way. For that to work reliably, however, it is necessary that nobody opening the file modify it in any way before obtaining a lock. open my $fh, '>', $lockfile modifies the file before there's an opportunity to lock it.

There are strategies for preventing this situation. One is to use sysopen for finer-grained control; open with O_CREAT but without O_TRUNC, for example. Another option is to open in append mode. I've used both. If opening in append mode, with the intent of clobbering the content of the file after obtaining a lock, it would be necessary to seek to the beginning and truncate manually after obtaining the lock, before writing into the file. One reason for using sysopen would be if you want the call to fail entirely if the file already exists: O_CREAT|O_EXCL.

There's an MJD talk, the slides for which are really useful: File Locking Tips and Traps. In those slides, "+<" is used.

I think I gave a YAPC talk on flocking a few years back that may also provide some explanation:

Anyway, the point to this follow-up is please don't open in '>' mode and then obtain a lock, if you care about the contents of the file, because if you fail to get the lock because someone else holds it, you just stomped on them.


Replies are listed 'Best First'.
Re^2: File lock demo
by LanX (Saint) on Apr 21, 2021 at 19:44 UTC
    Thanks, very informative!!! :)

    In my case I'm indeed logging the PID, but as a name of another file plus timestamp. And I'm indeed writing content into that other file.

    My issue is that Windows (or maybe the way flock is ported to Win) doesn't allow me to read a file which is "partially locked" by Perl. (at least I couldn't figure out how)

    Hence the content is only readable if no script is running.

    This is most probably different on Linux but caused me much headache to debug and led me to the shown solution with an empty lock-file.

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

      Empty lock files certainly aren't wrong either. MJD's talk discusses how many problems can be solved by using a lockfile that is different from the target working files. And that's what you're doing. I mostly just didn't want someone seeing open my $fh, '>', ...; and then flock, and think this idiom will translate well to locking an output file that they intend to write to.


        > and think this idiom will translate well to locking an output file that they intend to write to.

        Yeah, I have to make that clear to my colleagues. My concern was the warning in the docs that locking might fail if the lock-file is not opened for writing.

        And the side effects on Win are giving me a hard time when following recipes born on Linux.

        Thanks for sharing those links! :)

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery

Re^2: File lock demo
by LanX (Saint) on Apr 24, 2021 at 00:33 UTC
    Hi Dave

    I finally found the 45 min to watch the video. :)

    I've actually been at this conference, pity I missed your talk.

    I just wanted to add for the records that unfortunately your advise that

    "you can unlink an open filehandle"

    is not universally true. :/

    This fails on Win:

    > perl print my $file="random".rand(1000); open my $fh,">",$file or die $!; unlink ($file) or die $! __END__ Permission denied at - line 1. random741.284161646949

    The same code runs without issues on linux

    Thanks again! :)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

      Thanks, that's good information. And how unfortunate, too. Every OS has its warts. I guess I'm happy with the devil I know (Linux).


        > And how unfortunate, too.

        Thankfully I don't have the requirement for an "atomic deletion" of the file, because

        • I'm locking a semaphore file
        • the target files have unique names
        • their content is not security relevant
        • it's basically logging for observing state and retrieving errors

        But I'm trying to build my solution as OS agnostic as possible.

        I will let you know if I find something for "atomic deletion" on Win.


        ) though a use case might help to wrap my head around it. And Win has symlinks now...

        > I guess I'm happy with the devil I know (Linux).

        I still rather feel like a Linuxer even after 5 years in a Win project.

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery

        > This reminds me of a long battle I had with The Damian,

        For the records, no battle here. Davido only mentioned this after being asked, that was not part of any slides. :)

        > he made a boo boo

        FWIW I always thought that documentation should have a test suite too.

        At least the perldocs.

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery