Search for oldest file in directories and its subdirectories

p_boarder has asked for the wisdom of the Perl Monks concerning the following question:

Dear Master

I have multiple directories to read either the directory itself or recursive.
If files are found, I seek for the oldest one in that directory. If its a directory I should read the content of that directory and find the oldest file again. Age of file could be individual for each directory (entry point).

My script does the following steps:
a) read config

b) prepare config
- put "non recursive directories" and their age in an array
- put "recursive directories" and their age in another array

c) foreach element of "non recursive array" do a search for the oldest file using sort. Compare oldest file with the given age and write to log file.
d) foreach element of "recursive array" do a search for?
At this point I get stuck. How can I finish this script?

I'm not allowed to use any non standard modules.
Thank you for your answers in advance.
Your respectful scholar
Here's the code

##############################
# Pragmas
##############################
use strict;
use Sys::Hostname;
use File::Basename;
use File::Find;
use File::Copy;
use diagnostics;
use Data::Dumper;

#--------------------------------------------------
##############################
# Variables, Hashes and Arrays
##############################
my $configfile = qq(/anyDir/anyUser/dat/anyConfigfile);
my @dirsToInvestDirAge;
my @dirsToInvestDir;
my @confContentSplit;
my @confDirNonRecursive;
my @confDirRecursive;
my @foundDirsFromRecursive;
my @foundFilesFromNonRecursive;
my @foundFilesLastRun;
my @filesFromNonRecursive;
my $CheckLogFile = qq(/anyDir/anyUser/log/anyLogfile);
my $thisHost = hostname();
my $currTime = time();
my $tmpCurrentRun = qq(/anyDir/anyUser/log/anyCurrentRun.tmp);
my $tmpLastRun = qq(/anyDir/anyUser/log/anyLastRun.tmp);

#--------------------------------------------------
##############################
# Subs
##############################
sub getLogDateTime
{
        # do all the requested time formats
        my $formatParm = shift();
        my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isds
+t) = localtime(time);
        $year += 1900;
        $mon += 1;
        if ( length($mon) < 2 )         { $mon = "0".$mon};
        if ( length($mday) < 2 )        { $mday = "0".$mday};
        if ( length($hour) < 2 )        { $hour = "0".$hour};
        if ( length($min) < 2 )         { $min = "0".$min};
        if ( length($sec) < 2 )         { $sec = "0".$sec};

        if ( $formatParm eq 'log' )
        {
                return "$year-$mon-$mday $hour:$min:$sec";
        }
        if ( $formatParm eq 'logext' )
        {
                return "$year$mon$mday$hour$min$sec";
        }
}

#--------------------------------------------------
# Prepare configuration
sub prepareConfig
{
        # read and prepare the configuration
        open (CONFIG,"<$configfile") or die qq(cannot open configurati
+on file: $!\n);
        my @confContent = <CONFIG>;
    close(CONFIG);

        foreach my $line ( @confContent )
        {
                # no empty lines
                next if $line =~ s/^\s+//;
                # no empty lines
                next if $line =~ s/^\s*#//;
                # no lines with newline
                next if $line =~ s/^\n*#//;
                # add all found lines to array
                push(@dirsToInvestDirAge, $line);
        }
        #print qq(dirsToInvestDirAge: @dirsToInvestDirAge\n);

        # Write log file message config read
        print ANYLOGG "@{[getLogDateTime('log')]} AgeCheck $thisHost I
+NFO AgeCheck Config file read successfuly\n";
}

#--------------------------------------------------
sub findFilesRecursivCheckAge
{
        my $age;
        my @directoryListingForRecursive;

        # open current run temporary file to store found files for lat
+er comparsion
        openCurrRun();

        # Process every line of config array
        # First check for entries configured as recursive (R). If foun
+d add to array confDirRecursive,
        # else add to array confDirNonRecursive
        foreach my $lineDirAge ( @dirsToInvestDirAge )
        {
                #print qq(lineDirAge: $lineDirAge\n);
                # search for directories with recursive flag set (rege
+x digit, whitespace, letter "R"
                if ( $lineDirAge =~ /\d+\s+R$/ )
                {
                        push(@confDirRecursive, $lineDirAge);
                }
                else
                {
                        push(@confDirNonRecursive, $lineDirAge);
                }
        }

        # run runSearchDirsRecusiv with dirnames as argument from all 
+directories configured for recursiv search
        if ( @confDirRecursive )
        {
                foreach my $dirRecursive (@confDirRecursive)
                {
                        my @splitDirRecursive = split(/\s+/, $dirRecur
+sive);
                        my $ageNonrecursive = $splitDirRecursive[1];
                        #print qq(arrR0: $splitDirRecursive[0]\n);
                        #print qq(arrR1: $splitDirRecursive[1]\n);
                        #print qq(arrR2: $splitDirRecursive[2]\n);

                        #???????????????????

                }
        }
        #print qq(confDirNonRecursive: @confDirNonRecursive\n);


        if (@foundDirsFromRecursive )
        {
                #foreach my $
                #print qq(now doFileSearchForEveryDirRecursive\n);
        }


        # run runSearchFilesNonRecusiv with dirnames as argument from 
+all directories configured for non recursiv search
        if ( @confDirNonRecursive )
        {
                foreach my $dirNonRecursive (@confDirNonRecursive)
                {
                        my @splitDirNonRecursive = split(/\s+/, $dirNo
+nRecursive);
                        #print qq(arrN0: $splitDirNonRecursive[0]\n);
                        #print qq(arrN1: $splitDirNonRecursive[1]\n);
                        #print qq(arrN2: $splitDirNonRecursive[2]\n);

                        runSearchFilesNonRecusiv($splitDirNonRecursive
+[0],$splitDirNonRecursive[1]);
                }
        }

        closeCurrRun();
}

#--------------------------------------------------
sub runSearchDirsRecusiv
{
        my @params = @_;
        my $dirParam = $_[0];
        my $ageParam = $_[1];
        opendir (RECURDIR, $dirParamA);
        my @dirContent = readdir RECURDIR;
        closedir RECURDIR;
        print qq(foundDirsFromRecursive: @foundDirsFromRecursive\n);
}

#--------------------------------------------------
sub runSearchFilesNonRecusiv
{
        my $nonRecursiveDirName = $_[0];
        my $nonRecursiveAge     = $_[1];
        print qq(currDir: $nonRecursiveDirName\n);

        opendir(NONRECURSIV, $nonRecursiveDirName) or die "error while
+ opendir: $!\n";
               
        @foundFilesFromNonRecursive = grep {-f "$nonRecursiveDirName/$
+_" } readdir NONRECURSIV;
       
        close NONRECURSIV;
        print qq(filesFromNonRecursive: @foundFilesFromNonRecursive\n)
+;
        #print Dumper (@foundFilesFromNonRecursive);

        if ( @foundFilesFromNonRecursive )
        {
                foreach my $foundDirNonRec ( @foundDirsFromRecursive )
                {
                        #fileTimeComparsion($foundDirNonRec, $nonRecur
+siveAge);
                        print qq(now search for files from non recursi
+v in dir $nonRecursiveDirName\n);
                }
        }
        #undef @foundFilesFromNonRecursive;

}

#--------------------------------------------------
sub fileTimeComparsion
{

        my $absoluteFileName = $_[0];
        print qq(absoluteFileNamefileTimeComparsion: $absoluteFileName
+\n);
        my $ageToCheck = $_[1];

        # print files found to temporary file for later comparsion
        print CURRTMP "$absoluteFileName\n";

        my @timeStamp;
        my $timeStamp;
        my $currRunTimeStamp;

        # convert the given minutes into seconds
        my $AgeOfFile = ($ageToCheck * 60);

        # get age in epoc time for every file
        @timeStamp = stat($absoluteFileName);
        $currRunTimeStamp =  $timeStamp[8];
        #print qq(currRunTimeStamp: $currRunTimeStamp\n);

        # Deduct time found from current time in epoc seconds
        my $fileAgeSec = $currTime - $currRunTimeStamp;

        if ( $fileAgeSec > $oldestInitValue )
        {
                $oldestInitValue = $fileAgeSec;


                if ( $fileAgeSec > $AgeOfFile )
                {
                        #print qq(file: $absoluteFileName\n);
                        #print qq(dir: $File::Find::dir\n);

                        # Calculation of age of file in second back in
+ to day, hour, minute and second
                        my @parts = gmtime($fileAgeSec);
                        #printf ("%4d %4d %4d %4d\n",@parts[7,2,1,0]);

                        my $second      = $parts[0];
                        my $minute      = $parts[1];
                        my $hour        = $parts[2];
                        my $day         = $parts[7];


                        # Format all values to 2 digits
                        if ( length($second) < 2 )              { $sec
+ond = "0".$second};
                        #print qq(second: $second\n);
                        if ( length($minute) < 2 )              { $min
+ute = "0".$minute};
                        #print qq(minute: $minute\n);
                        if ( length($hour) < 2 )                { $hou
+r = "0".$hour};
                        #print qq(hour: $hour\n);
                        if ( length($day) < 2 )                 { $day
+ = "0".$day};
                        #print qq(day: $day\n);

                        my $timeStampBackCalc = qq($day day(s) $hour h
+r(s) $minute min(s) $second sec(s));

                        # Write error message only if file was not det
+ected in an erlier run

                        # check for files found in last run
                        my $filesFoundLastRun = basename($absoluteFile
+Name);
                        my @foundFilesLastRunTrue = grep(/$absoluteFil
+eName/, @foundFilesLastRun);
                        if ( @foundFilesLastRunTrue )
                        {
                                print ANYLOG "@{[getLogDateTime('log')
+]} AgeCheck $thisHost WARNING AgeCheck Age of $absoluteFileName is > 
+$AgeOfFile sec\; Age of file: $timeStampBackCalc\n";
                        }
                        else
                        {
                                print ANYLOG "@{[getLogDateTime('log')
+]} AgeCheck $thisHost WARNERR AgeCheck Age of $absoluteFileName is > 
+$AgeOfFile sec\; Age of file: $timeStampBackCalc\n";
                        }
                }
        }
}

#--------------------------------------------------
sub openCurrRun
{
        open(CURRTMP, ">>$tmpCurrentRun") or die "Can't open tmpCurren
+tRun file: $!\n";
}

#--------------------------------------------------
sub closeCurrRun
{
        close CURRTMP;
}

#--------------------------------------------------
sub readLastRun
{
        open(LASTTMP, "<$tmpLastRun") or die "Can't open tmpLastRun fi
+le: $!\n";
        @foundFilesLastRun = <LASTTMP>;
        close LASTTMP;
        #print qq(foundFilesLastRun: @foundFilesLastRun\n);
}

#--------------------------------------------------
sub openLog
{
        open(ANYLOG, ">>$CheckLogFile") or die "Can't open logfile: $!
+\n";
}

#--------------------------------------------------
sub closeLog
{
        close ANYLOG;
}

#--------------------------------------------------
sub moveTempFiles
{
        move($tmpCurrentRun, $tmpLastRun);
}

#--------------------------------------------------
############################
# Main
############################
openLog();
moveTempFiles();
readLastRun();
prepareConfig();
findFilesRecursivCheckAge();
closeLog();
[download]

Config file:

#Absolute path name                 #time in minutes        # recursiv
+e/ normal
/anyApp/path1/Dummy/dataDir/      30                  N
/helloApp/noPath/noName/nothing/  15                      N
/niceDayApp/guru/perlMonks/      30                      R
[download]

Comment on Search for oldest file in directories and its subdirectories Select or Download Code

Replies are listed 'Best First'.
Re: Search for oldest file in directories and its subdirectories by ambrus (Abbot) on Nov 03, 2009 at 11:32 UTC
See also Re: finding top 10 largest files.	[reply]
Re: Search for oldest file in directories and its subdirectories by Ratazong (Monsignor) on Nov 03, 2009 at 12:06 UTC
Hi! You might want to change your algorithm .. no need to store all directories to arrays. Try the following 1) create 2 vars (`$oldest_name; $oldest_age`) and initialize the age to 0 2) pass through all files using File::Find (e.g. `find(\&checkfile, $path);`) - you'll need `File::Find` for this 3) in your callback-sub (e.g. `checkfile`) check if the current file is older than the oldest you found till now; if yes, store the data to your `$oldest`-variables (hint: `File::stat` might be helpful here) now (when `find()` is finished) you have found the oldest file :-) HTH, Rata	[reply] [d/l] [select]
Re: Search for oldest file in directories and its subdirectories by rovf (Priest) on Nov 03, 2009 at 11:32 UTC
I have some trouble understanding your question, so it might be helpful if you could also post the code you have written, and not only an English description. Compare oldest file with the given age What do you mean by this? What age have you "given", and how do you compare a file with an age? Do you mean: compare two files? Do you mean: compare the timestamps of two files? If I understand you write, you have a directory, which you need to be sorted according to the timestamp, and log the oldest entry. If this entry happens to be a directory too, the procedure should be performed recursively to this directory as well, and so on. Is this correct? In this case, I don't see why you need two arrays. -- Ronald Fischer <ynnor@mm.st>	[reply] [d/l]
Re: Search for oldest file in directories and its subdirectories by stefbv (Priest) on Nov 04, 2009 at 06:40 UTC
There is a module on CPAN named File::Find::Rule::Age	[reply]
Re^2: Search for oldest file in directories and its subdirectories by rastoboy (Monk) on Nov 05, 2009 at 09:48 UTC
I would suggest that the requirement for no "non standard" modules should go. It makes no sense. One might think that that would enhance reliability and platform independence, but really "non standard" modules are just code--just like yours. If they can port your code around, they can port any modules that are needed. It might make sense to move them around with your code, however. Just sayin'.	[reply]
Re: Search for oldest file in directories and its subdirectories by vitoco (Hermit) on Nov 03, 2009 at 19:44 UTC
I use the following as a starting code for any recursive file task: Read more... (1236 Bytes)	[reply] [d/l]
Re: Search for oldest file in directories and its subdirectories by p_boarder (Novice) on Nov 18, 2009 at 16:19 UTC
Dear all, I'd like to thank to all who sent a reply related to this problem. The solution from vitoco fits at best. Special thanks to him. Regards p_boarder	[reply]