Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm working on some code that will be run on Red Hat Enterprise Linux 3, which removed support for flock() over NFS. I've been asked to remove all the flock() calls in this script and replace them with calls to fcntl(). I've tested both, and whenever flock() is called, $! gets populated with something like "No locks available". fcntl() seems to do locking properly, but has some performance issues. For instance, if process A has a lock of some sort on a file and process B comes along and tries to get a F_WRLCK (exclusive write lock) lock using F_SETLCKW, B hangs appropriately until A releases it's lock. But after A releases it's lock, it seems to take about 18 seconds for B to wake back up again. Using flock() to do the same scenario on a RedHat 7.3 machine, B wakes up almost immediately. I'm thinking maybe part of the problem lies in the way I'm calling fcntl(). From "man fcntl" on a RHEL3 system, I see that the packed buffer should look like:
struct flock { ... short l_type; /* Type of lock: F_RDLCK, F_WRLCK, F_UNLCK */ short l_whence; /* How to interpret l_start: SEEK_SET, SEEK_CUR, SEEK_END */ off_t l_start; /* Starting offset for lock */ off_t l_len; /* Number of bytes to lock */ pid_t l_pid; /* PID of process blocking our lock (F_GETLK only) */ ... };
Not being a C hacker, I'm not sure what the template for pack() in my fcntl() call should look like, since I don't see anything in the pod about off_t or pid_t. I've just been assuming that that off_t could be a long and pid_t an int, like so:
fcntl(FILEHANDLE, F_SETLKW, pack("sslli", F_WRLCK, SEEK_SET, 0, 0, 0)) +;
A co-worker suggested I also try:
fcntl(FILEHANDLE, F_SETLKW, pack("s!s!l!l!i!", F_WRLCK, SEEK_SET, 0, 0 +, 0));
I'm not even sure what the bangs do exactly, but this didn't seem to help any. Am I creating the packed buffer correctly? If so, is there anything that can be done to help the performance? 18 seconds is way, way too slow.

Replies are listed 'Best First'.
Re: Pack and off_t for fcntl()?
by doowah2004 (Monk) on Sep 20, 2004 at 13:37 UTC
    I read this somewhere (not tested, but makes since), fcntl in notorious for perfomance issues, but in your case I think that you can get around this by not allowing F_SETLKW to do the lock checking. Instead if you used F_GETLK it may speed things up. Also you may try SYSVSEM2 instead of fcntl, this paper outlines some of the issues well.

    Hope that this helps,

    Cameron

    NOTE: For the most part I am talking way over my head, so I could be very wrong with everything that I said, please take take my words as goodfaith suggestions and maybe a point in a better direction.
Re: Pack and off_t for fcntl()?
by waswas-fng (Curate) on Sep 20, 2004 at 14:59 UTC
    Once a file is locked over NFS, it turns off all local memory caching on the file (which means every thing that is done to the file now happens over the remote NFS link -- read slow) until _all_ processes that had the file open close (even after the lock has been released). You may be seeing an 18 second delay because the file is large and NFS is using network mode instead of local cache mode to open and read the data. How big is the file and how long does it take to copy the entire file to the server the B process is running on? This is one of the major issues with NFS that vendors like NetAPP would have you ignore.


    -Waswas
      It doesn't seem to matter what size file I'm using--even just a few dozen bytes takes 18+ seconds to get the lock. =/

      These are very fast servers with gigabit ethernet links and very light load CPU/memory/network load, by the way, so I don't think resource constraints are a problem.