Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Need something to tell me if a file (actually a word document) has been added to a directory. I want to run this in a cron job where it runs twice a day and only sends me a message when a new document has been added to the directory. Here is an attempt as I would like to do this in a hash but cant get it to work. I know I need some sort of work with "oldfiles" set some other way and sending a message if I get a new file. Please help me on this??
#!/usr/local/bin/perl my $mydir="/path/dirname"; use strict; opendir(DIR,$mydir) || die "Could not open $mydir: $!\n"; my %files=map {$_,1} readdir(DIR); my %newfiles = ""; foreach my $file (keys %newfiles) { unless (exists $oldfiles{$file}) new_file(); }

update (broquaint): title change (was NOtification of a new file added to directory)

Replies are listed 'Best First'.
Re: Notification of a new file added to directory
by gjb (Vicar) on Jan 03, 2003 at 14:17 UTC

    Why not use the -M operator to test for the age of the file (result in days). So something like:

    sub test_new { my ($mydir) = @_; my $new_one = 0; opendir(DIR, $mydir) or die("..."); while (my $file = readdir(DIR)) { if ($file !~ /^\.+$/ && -M $mydir . '/' . $file < 0.5) { $new_one = 1; last; } } closedir(DIR); return $new_one; }
    would do.

    Hope this helps, -gjb-

    Update: Stupid me! I tested the script only in the current working directory, obviously the file test should read -M $mydir . '/' . $file < 0.5 rather than -M $file < 0.5 which looks for the file in the script's directory. (modified the original code already). Incidently, the error is easily caught with use warnings; since a warning about the use of an uninitialized value in numeric lt is issued.

    Update: Added code to filter out the . and .. directories that were tested as well.

      While this will work fine if the files are CREATED there, it may fail badly it they are MOVED. At least if they are moved within one logical disk.

      I'd use a hash tied to a DBM containing the last modification dates of all files in the directory and processed all that 1) were not present or 2) have a different modification time.

      use DB_File; tie %ages, 'DB_File', $age_database, O_CREAT|O_RDWR, 0700; sub test_new { my ($mydir) = @_; my $new_one = 0; opendir(DIR, $mydir) or die("..."); while (my $file = readdir(DIR)) { next if $file !~ /^\.+$/; my $mtime = (stat($file))[9]; next if $mtime == $ages{$file}; $ages{$file} = $mtime; #the $file is new! Process it any way you like $new_one = 1; } closedir(DIR); return $new_one; }

      If all you need to know is whether there was any change then maybe it would be better to generate an MD5 hash of the directory contents (filenames + their modification times) and compare the result with a stored value.

      Jenda

      I tried this and it seems to return "true" (1 value) on my directory even though last added file is a week old. Please advise what I am doing wrong?
      #!/usr/local/bin/perl use strict; my $mydir = "/mydirectorypath/directory"; my $new_one = 0; opendir(DIR, $mydir) or die("..."); while (my $file = readdir(DIR)) { if (-M $file < .5) { $new_one = 1; last; } } closedir(DIR); print "$new_one\n";

        I modified the original node. Sorry for the confusion, it was silly on my part, -gjb-

      It still returns a "1" on my directory even if files are older than a day. Is there something I am doing wrong here???
      #!/usr/local/bin/perl use strict; my $mydir = "/mypath/directory"; my $new_one = 0; opendir(DIR, $mydir) or die("..."); while (my $file = readdir(DIR)) { if (-M $mydir . '/' . $file < 0.5) { $new_one = 1; last; } } closedir(DIR); print "$new_one\n";

        I updated the code again, see if this helps.

        Incidently, why not register, it's much easier to notify you of a change using the chatterbox rather than having to post these otherwise useless notes.

        Hope it works for you this time, -gjb-

Re: NOtification of a new file added to directory
by Three (Pilgrim) on Jan 03, 2003 at 14:31 UTC

    Heay I wrote a program that deals with detecting changes in a file system.

    Check out my Program Archiver. Specifically check out the make_comp subroutine.

    The basics of how it works is that it gets the current files in a subdirectory.

    Then  it reads a file that has the previous directory contents. 

    Then it generates a comparison so it can know if a files is new, changed, unchanged, or deleted.

    Hope this helps.

Re: Notification of a new file added to directory
by waswas-fng (Curate) on Jan 03, 2003 at 17:31 UTC
    One more thing of note here is that to avoid race conditions you may want to come up with some kind of locking system for the file as they are being written, or you may inadvertantly grab a partially written file for whatever new_file(); does. If you can't control the creation of the files then you may want to do a more complicated test over X time to determine that the file is new _ AND _ that the file is not currently changing before sending to the sub. This second approch still leaves a race situation - but it seems to be a little tighter than just assuming that a new files is not being modified. Also if this is on Win32 you can use one of the API modules on CPAN to open the file with Native locking.

    -Waswas
Re: Notification of a new file added to directory
by kschwab (Vicar) on Jan 03, 2003 at 15:55 UTC
    This might seem like a big hammer for a small problem, but check out AIDE.

    It's a tripwire like ( minus the silly licensing ) utility that will checksum files and directory trees and let you know about all changes. It's also configurable enough to ignore changes you aren't interested in.

    Sample output:

    AIDE found differences between database and filesystem!!
    Start timestamp: 2002-09-23 03:30:00
    Summary:
    Total number of files=18158,added files=1,removed files=1,changed files=0
    
    Added files:
    added:/etc/rc3.d/S7snmpdx
    Removed files:
    removed:/etc/rc3.d/s76snmpdx
    
    End timestamp: 2002-09-23 03:30:04