There is always a risk that when you take a snap-shot of an active data, that by the time you get around to using some of it, it could be out of date.

A couple of possibilities to work around this

If your not concerned with dealing with new stuff that has appeared since you snapshot was taken, then use -e to verify that files/dirs still exist before you take any action on or as a result of the presence of a file reported by File::Find.

If your running on a Windows system, you can register your interest in the (sub)tree that you have run File::Find on with Win32::ChangeNotify and periodically check for and record any changes that occurred since you did your File::Find. This complicates your code somewhat as every time you process a file, you would need to check your changes mechanism (a hash seems a possible choice) to see if the file in question has had any changes recorded against it.

There may be a similar mechanism in *nix, but I am not aware of it.

Without knowing quite what use you are making of the information returned by File::Find, its difficult to make any further suggestions.

One possible way to fix this would be to change the permissions on the file, compress, and then change the permissions back.

It is unlikely that someone will access an old file at the same time that your program compresses it, but better to prevent the possibility altogether.

It should work perfectly the first time! - toma

Changing the permissions would only work if you have your own user for the script -- say, you chown the file to yourself, chmod it to 600 and compress it, then change everything back. Why not just rename it? That would be atomic, plus if the renaming fails because the file or directory doesn't exist anymore, you can skip the compression step.

Hang about ! , if I have this correct - you want to traverse an NFS tree and action files that match your logic, but you are concerned about the FS changing under your feet. Is it possible you could nibble away at the output of whichever finder mechanism you choose, thus diminishing (hopefully) the time between 'knowing' about a file and actioning it?


This is off the cuff code- OK!


open ( FIND , '|' , '/usr/bin/find' , @findargs ) || die $!;
while ( <FIND> ) {
 
 if ( $_ =~ /$myfileMatch/ ) {
    if ( -M $_ > 14 ) {  # file modified more than 14days ago
        # delete file magic
    }
    else { # compress file magic }
 }

}
[download]

This way open is happily running find forked, whilst the script does dirty work. Of course if you are compressing monster files across NFS from a filer there is going to be a performance hit of somekind.

Of course this may not be what you want, and doing it this way I strongly recommend the script keeps a record of exactly what the hell it's doing.

Update: Since you are concerned about files that are ageing as opposed to new files, it perhaps does not matter too much that new data is added to this tree as the script runs. Reading back methinks that it's better to build a static list of matching files to operate on, then run through that list at the end of search, checking that they indeed -e exist.



I also hope you only have to deal with NFS to the filer, coz SMB op-locks might cause you pain

Please, can you explain better your situation? For example, what kind of file system are you using? has it quota support? how frequently would you like to update your thing? what kind of update do you need to perform?

Thanks, Valerio

Unless you come up with a traversal method that you specifically want to use because it works better for you, go ahead and use the module.