The problem with fuser (or lsof) is that it'll tell you the process has an open descriptor for the file, but that doesn't say anything about if it's actively writing to it. You could use something like SGI::FAM to tell if the file's being modified and then do your work when a long enough window of non-activity's passed, but that's not much better.

If the logging program can handle it (i.e. you can establish some window you can shut it down without worrying about losing records) it may be simpler to just: stop the logger, do whatever processing you're wanting to (or make a copy of the file), and then restart the logger.

The problem with fuser (or lsof) is that it'll tell you the process has an open descriptor for the file, but that doesn't say anything about if it's actively writing to it.

Detecting wether some process has an open fd for the file is fine and all that is needed for this particular task. Thank you for the additional insightful info provided, in any case.

Note this only works if you have access to the /proc entries of the process that is doing the access.

It would only work to show other accesses by your own userid unless you are root.

Particularly, when a process might reopen the file and append something, you have the advantage of having your file to work with and the process starts to write a new file.

Update:

Nope that is not true in the general sense, which can be demonstrated simply by running:

open FOO, ">x";
$foo = <>;
[download]

in one window and then in another:

[jonathan@orpheus test]$ /sbin/fuser -u x
x:                    5665(jonathan)
[jonathan@orpheus test]$ mv x x1
[jonathan@orpheus test]$ /sbin/fuser -u x1
x1:                   5665(jonathan)
[download]

However it is true for most Windows applications.

/J\

You can rename, or even delete a file while it is open, on Linux, *BSD, and similar OSes. That's one way to create a temporary anonymous file, which will disappear as soon as your open handle to it closes. It is caused by the strong dissociation between file name and file contents. Hard links are another example, where you can have two independent and completely fully qualified (no limits) paths to the same file contents.

Also, if on those platforms, you attempted to rename a log file for for example Apache while it is running, you'll see it doesn't stop it from still growing.

Windows won't allow you to rename or delete the file while its open, that's true.

A lot depends on what is producing the log file. Is using file test operators or stat() a possibility? If the last modification date on a log file is over a threshold value, can you assume the process has stopped writing to it?

A trick I used to use with Oracle log files was to rename the file while it was still open. You could then work on it.

Good luck!
Kathy

A lot depends on what is producing the log file. Is using file test operators or stat() a possibility? If the last modification date on a log file is over a threshold value, can you assume the process has stopped writing to it?

Actually we've been thinking about this ourselves, and despite what I wrote in the first node of this thread, I suppose that eventually we'll either use that or run fuser externally rather than processing /proc ourselves (or we'll check if Linux::Fuser works with current kernels -I suppose so, BTW- and if so, then use that instead).

or we'll check if Linux::Fuser works with current kernels

It definitely works on x86 2.6 Linux kernels but if you have a problem with it drop me a line indicating the kernel version and CPU architecture and I'll fix it

Update: I have just uploaded a new version of Linux::Fuser which includes an example program and is more explicit in the README about what kernel versions it is known to work with.

/J\

I guessed when I wrote Linux::Fuser and looked in /proc/*/fd and found all the file descriptors that had the same inode as the file I was interested in. Having just looked the source of fuser it appears it was a good guess ;-)

/J\

Is this a proper assumption? From what I know, inodes are unique based on filesystem, not on a system-wide basis. That is to say that you could have two file systems with two different files that have the same inode number.

thor

Feel the white light, the light within
Be your own disciple, fan the sparks of will
For all of us waiting, your kingdom will come

How does fuser do it?

I wish I knew. The manpage mentions /proc, maybe that could be relevant, but then if I knew I wouldn't have asked. And no, I've not checked the src, as it is C, I suppose, and I don't know much C any more...

But you can check the source of Linux::Fuser which works in the same way as I noted above in Re^2: How to detect if file is in use?.

/J\

In fact that is another option we've been taking into account. It depends on our ability to access the src code of the given program, which we actually have but which is not an actual option ATM for various reasons far too long (and probably irrelevant) to be explained here now.

However there's an issue with some instances of the log writing program dying unexpectedly (that's not strictly true, but it's the best approximation I can give without having to go into full irrelevant details) which is precisely one of the things we're coping with. So after all this wouldn't be a viable option in our case.

But then you've transformed the problem into something completely different, but not necessarily more easily solved. The new question becomes how long must you wait to ENSURE that log file is not being appended to? One second may not be long enough, nor 10, 100, 1000 etc.

-Scott

Ignoring for a moment that some logfiles can remain open for several days without seeing a new entry -- what's the point in making a copy? Just do -s $file or (stat($file))[7] at t0 and t1..tn and save some disk space. It won't tell you if the file is open, but it will tell you if it's changing while you're looking at it.