Monks,

I have a script that scans an inputed list of user directory locations, and then reports the name and size of the user directory. The information is output into a CSV file. At the end of my script, I call a program I wrote in VB that will input that CSV file into a table in a MS Access DB.

My problem is that my "output.csv" file is not complete. For some reason the script usually only gets about half of the information into the CSV file, although it completes without a hitch. I should mention that I am scanning a very large volume and the script often takes 20+ hours to run.

Would invoking a SLEEP function before I call my VB program improve this at all? Or the better question would be, what do I need to do to ensure that my entire output is recorded to my "output.csv" file? Here is my code:

#!/usr/bin/perl


use File::Find;
use strict;
use Time::localtime;
use File::Spec;

my $cur = File::Spec->curdir;
my $up  = File::Spec->updir; 
my $dirtot;

my ($dir, @parts, $error, $starttime, $runtime, $endtime, $runmin, $ye
+ar, 
$month, $day, $tm, $date, $rootdir, $userdir, $slash, $count, $runhour
+, 
$starttime, $runtime, $endtime, $runmin, $year, $month, $day, $tm, $da
+te);

$starttime = (time);
$slash = "//";

#Opens the input and output files
open(IN, "< input.txt") or die("Couldn't open input.txt\n"); 
open(OUT, "> output.csv") or die("Couldn't open output.csv\n");

#Separates the newline delimited rows
chomp ( @parts = <IN> );

#Used to calculate how long the program took to run
$starttime = (time);                    

#Prints headings
print OUT "Path,User,MB,Date,Error\n";

#Displays total for subdirectories
foreach my $start (@parts) {
  my @dirs = &find_subdirs($start);
  foreach my $dir (@dirs) {          
            
     $tm = localtime;
     $year = $tm->year+1900;
     $month = $tm->mon+1;
     $day = $tm->mday;
     $date = "$month-$day-$year"; 
      
     ($rootdir,$userdir) = split /$slash/,$dir;
     print "\tWalking $dir\n";
     my $total = 0;
     find sub { $total += -s}, $dir;
     $dirtot = $dirtot + $total;            #Used for determining tota
+l directory size
     $total = ($total / 1024) / 1024;        #Converts to MB
     $total = sprintf("%0.2f", $total);        #Formats output to 2 de
+cimals
     print OUT"$rootdir,$userdir,$total,$date,$!\n";
  }
  $dirtot = ($dirtot / 1024) / 1024;        #Converts to MB
  $dirtot = sprintf("%0.2f", $dirtot);        #Formats total to 2 deci
+mals
  print STDERR "$start: No subdirectories\n" unless @dirs;
}

$endtime = (time);
$runtime = $endtime - $starttime;
$runmin = $runtime/60;
$runmin = sprintf("%0.1f", $runmin);
$runhour = $runmin/60;
$runhour = sprintf("%0.1f", $runhour);

if ($runtime <= 60){print "\n\nCompleted processing in $runtime second
+s\n\n";}
elsif ($runmin <=60) {print "\nCompleted processing in ". $runmin . " 
+minutes\n\n";}
    else {print "\nCompleted processing in ". $runhour ." hours\n\n";}

    
# This closes output file and then calls an external executable that 
# will insert CSV output into an MS Access database table
close(OUT);
system "csv2access.exe";
print "\nOutput created.";



sub find_subdirs {
  my $start = shift;
  unless(opendir(D, $start)) {
     warn "$start: $!\n";
     next;
    
     }
  my @dirs =
     map {
         -d "$start/$_" &&
        !-l "$start/$_" &&
        $_ ne $cur && $_ ne $up
           ? "$start//$_" : ()
         }    
     readdir(D);
  closedir(D);
  @dirs;
}
[download]

Any ideas?

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.