breezykatt has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I am trying to figure out a solution (perl script) that would copy a file to another directory by the timestamp in which it was written for a certain day. Furthermore, this needs to be modular enough to handle different sets of directories running at different intervals. So for example, 3 different directories contain 3,000 files each. Directory 1 might copy a set of files over every 300 seconds. Directory 2 might copy a set of files over every 1800 seconds. So, all the file timestamps that fall in that particular range as compared to the current time should be copied over for processing. And this needs to run for 24 hours until all files have been copied. Below is where I am at now... I am able to read in files, get file timestamps, figure out intervals, but having trouble figuring out how to copy based on the current time and range in which to move the set of files. Thanks.
#!/usr/bin/perl use POSIX; $count_interfaces = `awk ' END { print NR } ' /MM/work/control/interfa +ce_frequencies.txt`; $count_interfaces = $count_interfaces - 1; print "count_interfaces = $count_interfaces\n"; chomp($count_interfaces); $starttime=localtime time; print "**********************Start Extracting Interfaces: STARTTIME=$s +tarttime**************************************\n"; removeInterfaceFile(); getDirectories(); getFiles($directory,$directory_count); calcIntervals($directory_count,$count_interfaces); $endtime=localtime time; print "*******************Finished Extracting Interfaces: ENDTIME=$end +time******************************************\n"; sub getCurrentTime { $now_string = strftime "%Y:%m:%d:%H:%M:%S", localtime; print "now_string = $now_string\n"; } sub getDirectories { $dir = "/MM/work/MMdayofdata"; $interface_directory="MMdayofdata"; $directory_count = 0; @directories = <$dir/*>; } sub getFiles { foreach $directory (@directories) { if (-d $directory) { print "INTERFACE_$directory_count=$directory\n"; $directory_count++; $subdir = $directory; @files = <$subdir/*>; ($dir1,$dir2,$dir3,$dir4,$dir5,$interface)=split(/\//,$dir +ectory); $interface =~ s/ //g; $interface =~ s/\n//g; $interface =~ s/\r//g; print "directory=$interface\n"; open(OUT3, ">/MM/work/control/$interface.dat"); #open(OUT3, ">>/MM/work/control/FILETIMESTAMPS.dat"); open(OUT3, ">>/MM/work/control/MMinound/"); foreach (@files) { my $file = "$files/$_"; $file =~ s/ //g; $file =~ s/\n//g; $file =~ s/\r//g; $file =~ s/\/\//\//g; my $mtime = (stat($file))[9]; $file_time = strftime("%Y%m%d_%H:%M:%S", localtime("$m +time")) . "\n"; $hour = strftime("%H", localtime("$mtime")) . "\n"; $minute = strftime("%M", localtime("$mtime")) . "\n"; #parse filename #get hour of day, minute of day to decide how to write + filename. #if the minute is 1-5 minutes, they fall into same cat +egory for 300 seconds #if (($hour == $nowhour) && ($minute == $nowminute)) { $subsubdir="/MM/work/control/$interface_directory/$int +erface/"; print "subsubdir=$subsubdir\n"; @interface_files = <$subsubdir/*>; print "interface_files = $interface_files[0]"; $reccount = 0; foreach $record (@interface_files) { chomp($record); #$record =~ s/*//g; #rename($record, "$interface_m_PT$hour$minute$ +reccount_00.dat"; $reccount++; } print OUT3 "$file\t" . "$file_time"; } close(OUT3); } } } sub calcIntervals { open(FH4, "</MM/work/control/interface_frequencies.txt") or die("C +ould not open interface_frequencies.txt"); open(OUT6, ">/MM/work/control/RUNTIME.dat") or die("Could not writ +e to RUNTIME.dat"); @Out4 = <FH4>; close(FH4); $count_rows = 0; #First record in OUT4 is a header row foreach $m_interface (@Out4) { chomp($m_interface); if ($count_rows == 0) { #Print out the hearder row for runtime.dat file print OUT6 "INTERFACE,FREQUENCY,TOTAL_INTERVALS_PER_DAY,FI +LES_PER_INTERVAL,TOTAL_FILES\n"; $count_rows++; next; } ($interface, $frequency)=split(/\|/,$m_interface); chomp($interface); chomp($frequency); open(FH5, "</MM/work/control/$interface.dat") or die("Could no +t open $interface.dat"); @Out5 = <FH5>; close(FH5); $total_files = `awk ' END { print NR } ' /MM/work/control/$int +erface.dat`; chomp($total_files); print "$interface = $total_files\n"; warn "Number of files for $interface is less than 1: $total_fi +les\n" if ($total_files < 2); $totalIntervalsPerDay = floor(86400/$frequency); warn "$interface: Number of intervals per day is less than 1: +$totalIntervalsPerDay\n" if ($totalIntervalsPerDay < 1); #print "$interface: Intervals Per Day: 3600/$frequency=$totalI +ntervalsPerDay\n"; $filesPerInterval = floor($total_files/$totalIntervalsPerDay); warn "$interface: Number of files per interval is less than 1 +_$filesPerInterval_ $totalIntervalsPerDay > $total_files\n" if ($file +sPerInterval <= 0); #print "Files Per Interval $interface: $total_files/$totalInte +rvalsPerDay=$filesPerInterval\n"; print OUT6 "$interface,$frequency,$totalIntervalsPerDay,$files +PerInterval,$total_files\n"; $count_rows++; } close(OUT6); } sub removeInterfaceFile() { print "Removing temp directories...\n"; `rm -rf /MM/work/control/RUNTIME.dat`; `rm -rf /MM/work/control/FILETIMESTAMPS.dat`; `rm -rf /MM/work/control/$interface.dat`; }

Replies are listed 'Best First'.
Re: copy or ftp file by file timestamp
by imrags (Monk) on Jul 23, 2009 at 04:49 UTC
    Try something like this:
    scalar localtime((stat("$dir_name/$file"))[9]);
    This will give the modified/created time of the file. If you are failing to browse thru the directory, then here's the code.
    print_dir ( "YOUR/DIRECTORY/PATH" ); sub print_dir { my ($dir_name) = @_; opendir ( my $dir_h , "$dir_name") or die "Unable to open dir :$dir +_name: $!\n"; while ( my $file = readdir($dir_h) ) { next if ( "$dir_name/$file" =~ /\/\.$/ or "$dir_name/$file" =~ /\ +/\.\.$/ ); if ( -d "$dir_name/$file" ) { print_dir ( "$dir_name/$file" ); } print "$dir_name/$file - "."\n"; } }
    It'll browse thru directories/sub-directories as well.
    Raghu
      Thank you for the response. I have not tried it out yet, but looks much tighter than what I wrote. I was able to get the timestamp of the files, the problem I am experiencing is how to copy a set of files to another directory, say every 300 seconds. So, for example, if you take the current time and read in the files in a particular directory, whatever files are in that 5 minute range should be copied over. The script needs to run for 24 hours straight until all files are copied or moved over. The next part is even more difficult. I need to make it modular enough to consider multiple directories. So for example, one directory might copy files over every 300 seconds while another directory might copy files over every 1800 seconds. Hope that makes sense? I do not know if I should make 1 perl script for each directory or make like 25 different perl scripts running at different intervals per directory. I'm having difficulty writing one script to do all.
        I might not be explaining this well....Basically - directory1 has 3,000 files in it with timestamps all throughout one day (July 23, 2009). directory2 has 5,000 files in it with timestamps all throughout one day (July 23, 2009). directory 1 should copy files over at a predefined interval rate, say 300 seconds. so if I can take a current start time say 9:00 am, and copy all files at 9:05 am that range from 9:00 am - 9:05 am. sleep for 300 seconds, then at 9:10 copy all files from 9:05 - 9:10, etc.... directory 2 follows the same methodology but runs at 1800 second intervals. the problem i am having is writing 1 perl script, not a bunch to do this. it doesn't have to be perfect, but just needs to do something along those lines.