ferrency has asked for the wisdom of the Perl Monks concerning the following question:

I discovered Parallel::ForkManager the other day, and rejoiced. Unfortunately, in using it I ended up tearing my hair out due to a somewhat unrelated problem: the interaction of flock() and dup'd file descriptors in child processes.
(perl 5.6, FreeBSD 4.1.1-stable, but the principles apply equally elsewhere)

The problem:

Why?
In retrospect, this is easy- straight out of the perldoc for flock(), and also in `man flock`:

On systems that support a real flock(), locks are inherited across fork() calls, whereas those that must resort to the more capricious fcntl() function lose the locks, making it harder to write servers.

I fall into the first category: since the lock exists per-file, and the filehandle only has a reference to the lock, and since flock()'ing a file you already have a lock on succeeds, the children all get locks whenever they ask for them.

My solution
This is also not hard. Instead of opening the file in the parent, and flocking then appending in the children, I perform a standard locking append in the child whenever I want to append to a file:

# Warning: untested.  If this were actual code, I would "use strict;"
open (F, ">>$my_file");
flock (F, LOCK_EX) or die "no lock!";
seek (F, 0, 2); # In case someone wrote after open but before I got th
+e lock
print F $my_stuff_to_print;
close F;

[download]

Why I don't like it
You can think of my particular case as similar to log file output. Multiple children need to append lines to one or more shared files without interrupting each other. Nothing is reading the files until it's known that all writing has completed and all child processes exit. I don't care about the order of the lines as long as no line is cut in half by any other line. In practice, the output is short enough not to cause any problems, but that's not good enough for those of us who are paranoid.

Because the reason I'm forking in the first place is to get better performance, I don't like the overhead of the open, flock, close calls every time I want to print a single short line to a file. Though I admit that this is a small portion of the total time spent in each child (as each child only outputs one line to any particular file).

Therefore,

The Question
Isn't there a better way? (read: more efficient than open, lock, close every time) If so, what is it?

Thank you for your time :)

Alan

Replies are listed 'Best First'.
(tye)Re: Those fork()ing flock()ers...
by tye (Sage) on Dec 05, 2001 at 02:11 UTC

    Sounds like you could avoid locking completely and just open the file for "append" access in the parent and use syswrite to ensure that each line is appended in a single I/O operation.

            - tye (but my friends call me "Tye")
[reply]
      This sounds like a good solution, and would probably do what I need.

      So, syswrite() and/or the underlying "write" system call is/are atomic?
      Isn't it possible for syswrite() to write less than the whole data it's told to write? If so, then how is this different than using auto-flush and writing with a print() call to the filehandle?

      (at some level, aren't you going to run into a size limitation which prevents write() from completing the write atomically due to hardware buffering or something? Or is that sort of like saying, "eventually it'll fail when the earth falls into the sun"?)

      Thanks. This might be just the solution I was looking for, but I don't think I understand syswrite() and the write() system call well enough to know for sure :)

      Alan

      Update:
      Looking in Programming Perl, I think this doesn't totally solve the problem. It gives me control in cases where a full write can't be done, but it doesn't prevent this from happening:
      You must be prepared to handle the problems that standard I/O normally handles for you, such as partial writes.

      (if it Happens To Work with syswrite, that may only be for the same reason it Happens To Work when I use stdio and no locking...)

[reply]

        Yes, appending is atomic. From "man 2 write":

        If the O_APPEND flag of the file status flags is set, the file offset will be set to the end of the file prior to each write and no intervening file modification operation will occur between changing the file offset and the write operation.

        A partial syswrite is possible, but, for the case of "regular" files, means that writing the rest of the data is going to fail anyway (unless the resource exhaustion that caused the initial partial write is resolved in the interim).

        I was about to update my node with the following alternative when I noticed your reply. Simply reopen the file once in each child and use flock as usual. The reason that flock doesn't work is because the file descriptors are all duplicates of each other. The documentation I was able to find on flock really sucked at explaining it (as far as I'm concerned, the Linux version was simply incorrect). But if that didn't work, then flock would be useless. (:

        (Updated to add "once" above.)

                - tye (but my friends call me "Tye")
[reply]
Re: Those fork()ing flock()ers...
by Rhandom (Curate) on Dec 05, 2001 at 03:40 UTC
    Do you write a log line multiple times during the forked process. If so, open the file once at the beginning, lock and unlock everytime you need to log a line, but still keep the file open. Even if you are only logging once, the overhead of forking is probably harder on resources than opening the file for appending. The nice thing is that appending will always jump to the end, even if somebody else has printed to the file since you opened the file.

    Another possible option is to open a pipe in the parent process, and fork off a child process that is only responsible for receiving the log. Upon thinking about this though, I guess you might have the same problem with possible co-mingling of lines while printing to the pipe buffer.

    So, to optimize your system, keep forked children alive as long as possible, and open the file to be locked first thing.

    my @a=qw(random brilliant braindead); print $a[rand(@a)];

[reply]
      Thank you, this is a good idea. I do have one "actual" logfile which may receive more lines than my various "result" files. In this case, since I always know I'll be logging, I can open that one once, and then only open the various "result" files when I have results to output, since I don't always use all of them in every child.

      The more I think about this, the more ridiculously picky I think I'm being :) With a "big calculation, small result" printing out the result is way cheaper than calculating it anyway, so I shouldn't really worry about the extra open's...

      Alan

[reply]
Re: Those fork()ing flock()ers...
by fokat (Deacon) on Dec 05, 2001 at 04:00 UTC
    And why don't you simply write more than one file (one per process)?

    I guess this might have even better performance (caching can occur between writes and log lines are definitely not going to be cut by each other).

    Later on, after processing is done, you can either merge all the files in a single one or process the individual files.

    Depending on your actual application, this approach might be faster than doing syswrite() each time.

    Good luck.

[reply]
      The fact that the answer to your initial question is, "Because it's easier the way I'm doing it now" tells me that I'm putting too much effort into squeezing a tiny speed increase out of this system :)

      Parallel::ForkManager is really good at controlling the maximum number of forked children, when you're trying to fork multiple children to solve parts of a large problem in parallel. Unfortunately, my particular problem has a Lot of medium-sized parts, instead of a few large parts (They each wait for network responses, but none do much in the way of calculation; I'd rather be waiting for 10 network results at a time instead of each one sequentially).

      Actually, now that I think of it, I could easily break my list of 10000 medium-sized problems into 10 lists of 1000 problems each, and feed each one to a child. This would be much more efficient than forking 10000 children, 10 at a time, each solving only one problem. And it would also be quite easy :)

      Thank you for your suggestion!

      Alan

[reply]
(MeowChow) Re: Those fork()ing flock()ers...
by MeowChow (Vicar) on Dec 05, 2001 at 06:14 UTC
    comment retracted on account of MeowChow being stupid and needing more sleep.
[reply]
      thanks a lot for making me use a vote on this node. i almost fell off my chair, it was so silly. i guess i could use a nap myself.

      ~Particle

[reply]

Back to Seekers of Perl Wisdom