Monks,

I have a script that scans an inputed list of user directory locations, and then reports the name and size of the user directory. The information is output into a CSV file. At the end of my script, I call a program I wrote in VB that will input that CSV file into a table in a MS Access DB.

My problem is that my "output.csv" file is not complete. For some reason the script usually only gets about half of the information into the CSV file, although it completes without a hitch. I should mention that I am scanning a very large volume and the script often takes 20+ hours to run.

Would invoking a SLEEP function before I call my VB program improve this at all? Or the better question would be, what do I need to do to ensure that my entire output is recorded to my "output.csv" file? Here is my code:

#!/usr/bin/perl


use File::Find;
use strict;
use Time::localtime;
use File::Spec;

my $cur = File::Spec->curdir;
my $up  = File::Spec->updir; 
my $dirtot;

my ($dir, @parts, $error, $starttime, $runtime, $endtime, $runmin, $ye
+ar, 
$month, $day, $tm, $date, $rootdir, $userdir, $slash, $count, $runhour
+, 
$starttime, $runtime, $endtime, $runmin, $year, $month, $day, $tm, $da
+te);

$starttime = (time);
$slash = "//";

#Opens the input and output files
open(IN, "< input.txt") or die("Couldn't open input.txt\n"); 
open(OUT, "> output.csv") or die("Couldn't open output.csv\n");

#Separates the newline delimited rows
chomp ( @parts = <IN> );

#Used to calculate how long the program took to run
$starttime = (time);                    

#Prints headings
print OUT "Path,User,MB,Date,Error\n";

#Displays total for subdirectories
foreach my $start (@parts) {
  my @dirs = &find_subdirs($start);
  foreach my $dir (@dirs) {          
            
     $tm = localtime;
     $year = $tm->year+1900;
     $month = $tm->mon+1;
     $day = $tm->mday;
     $date = "$month-$day-$year"; 
      
     ($rootdir,$userdir) = split /$slash/,$dir;
     print "\tWalking $dir\n";
     my $total = 0;
     find sub { $total += -s}, $dir;
     $dirtot = $dirtot + $total;            #Used for determining tota
+l directory size
     $total = ($total / 1024) / 1024;        #Converts to MB
     $total = sprintf("%0.2f", $total);        #Formats output to 2 de
+cimals
     print OUT"$rootdir,$userdir,$total,$date,$!\n";
  }
  $dirtot = ($dirtot / 1024) / 1024;        #Converts to MB
  $dirtot = sprintf("%0.2f", $dirtot);        #Formats total to 2 deci
+mals
  print STDERR "$start: No subdirectories\n" unless @dirs;
}

$endtime = (time);
$runtime = $endtime - $starttime;
$runmin = $runtime/60;
$runmin = sprintf("%0.1f", $runmin);
$runhour = $runmin/60;
$runhour = sprintf("%0.1f", $runhour);

if ($runtime <= 60){print "\n\nCompleted processing in $runtime second
+s\n\n";}
elsif ($runmin <=60) {print "\nCompleted processing in ". $runmin . " 
+minutes\n\n";}
    else {print "\nCompleted processing in ". $runhour ." hours\n\n";}

    
# This closes output file and then calls an external executable that 
# will insert CSV output into an MS Access database table
close(OUT);
system "csv2access.exe";
print "\nOutput created.";



sub find_subdirs {
  my $start = shift;
  unless(opendir(D, $start)) {
     warn "$start: $!\n";
     next;
    
     }
  my @dirs =
     map {
         -d "$start/$_" &&
        !-l "$start/$_" &&
        $_ ne $cur && $_ ne $up
           ? "$start//$_" : ()
         }    
     readdir(D);
  closedir(D);
  @dirs;
}
[download]

Any ideas?