How to time a directory scan

Someone had populated a single directory on our Sun file server with a couple million files, and I wanted to get some sense of the performance hit on reading the directory -- i.e. any sort of timing info on how long it would take to scan it all.

perl -le 'opendir(D,"huge_directory");
  while($_=readdir D) {
   $t = time()-$^T;
   $e++ if ( $t!=$prev );
   if($n and $e==15){ print $n; $n=$e=0 }
   $n++; $prev=$t
  }'
[download]

The idea was simply to report how many iterations of "readdir" would finish within each 15-second interval. This seemed to work, but it struck me as crude. Suggestions for improvement?

Comment on How to time a directory scan Download Code

Replies are listed 'Best First'.
Re: How to time a directory scan by atcroft (Abbot) on Aug 14, 2007 at 23:47 UTC
What about something like: `#!/usr/bin/perl use strict; use warnings; use Time::HiRes qw(gettimeofday tv_interval); my $testdir = $ARGV[0]; my @filelist; # Code example taken from Time::HiRes documentation opendir(D, $testdir) or die $!; my $t0 = [gettimeofday]; @filelist = readdir(D) or die $!; my $t1 = [gettimeofday]; closedir D; my $t0_t1 = tv_interval $t0, $t1; print q{Scan time for directory }, $testdir, q{ : }, $t0_t1, qq{ seconds\n}; print q{Total entries in directory }, $testdir, q{ : }, scalar @filelist, qq{\n};` [download] Example output: `$ perl readdir-test.pl . Scan time for directory . : 3.5e-05 seconds Total entries in directory . : 4 $ perl readdir-test.pl test/ Scan time for directory test/ : 0.113722 seconds Total entries in directory test/ : 68115 $ perl readdir-test.pl test2/ Scan time for directory test2/ : 1.475231 seconds Total entries in directory test2/ : 412483` [download] HTH. Updated: 14-Aug-2007 Added example for directory test/ containing 412483 entries.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: How to time a directory scan
by atcroft (Abbot) on Aug 14, 2007 at 23:47 UTC

What about something like:

#!/usr/bin/perl

use strict;
use warnings;
use Time::HiRes qw(gettimeofday tv_interval);

my $testdir = $ARGV[0];
my @filelist;

# Code example taken from Time::HiRes documentation
opendir(D, $testdir) or die $!;
    my $t0 = [gettimeofday];
    @filelist = readdir(D) or die $!;
    my $t1 = [gettimeofday];
closedir D;

my $t0_t1 = tv_interval $t0, $t1;

print q{Scan time for directory }, 
  $testdir, 
  q{ : },
  $t0_t1,
  qq{ seconds\n}; 
print q{Total entries in directory },
  $testdir,
  q{ : },
  scalar @filelist,
  qq{\n};
[download]

Example output:

$ perl readdir-test.pl .
Scan time for directory . : 3.5e-05 seconds
Total entries in directory . : 4
$ perl readdir-test.pl test/
Scan time for directory test/ : 0.113722 seconds
Total entries in directory test/ : 68115
$ perl readdir-test.pl test2/
Scan time for directory test2/ : 1.475231 seconds
Total entries in directory test2/ : 412483
[download]

HTH.

Updated: 14-Aug-2007
Added example for directory test/ containing 412483 entries.

[reply]
[d/l]
[select]