How do I count all the files in all subdirectories?

Marcello has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

I have a problem, I currently do this to count all the files in all available directories (a directory ONLY contains files, no subdirectories):

my @dirs = dir_entries($dir);
foreach $file (@dirs) {
    my @batches = dir_entries($dir.$file);
    my $amount = @batches;
    $total += $amount;
}

sub dir_entries {
    my ($dir) = @_;
    opendir(DIR, $dir);
    my @entries = readdir(DIR);
    close(DIR);
    # Remove . and ..
    @entries = grep (!/^\./, @entries);
    @entries = sort {$a cmp $b} @entries;
    return @entries;
}
[download]

This works fine, but there is a little something extra. The directory names are changed very rapidly, so when the script tries to read the contents of a directory, it can already be renamed, and therefore this directory is skipped.

Anyone has a solution to prevent this from happening?

Comment on How do I count all the files in all subdirectories? Download Code

Replies are listed 'Best First'.
Re: How do I count all the files in all subdirectories? by stefp (Vicar) on Sep 18, 2001 at 00:49 UTC
I assume you are using some kind of UN*X. Names are a thin layer on top of file system mechanism. Once you have a dirhandle open, one can change the names of the directory under you. This will not be a problem anymore. `use IO::Dir; my @dirhandle = map { IO::Dir->new($_) } @dirs;` [download] Now you got the dirhandles and can work without fear of directory renaming. I am not sure that you mean that only the dirs are changing name under you. If files are renamed, their inodes identify them uniquely. You can get these inodes using stat(). -- stefp	[reply] [d/l]
Re: How do I count all the files in all subdirectories? by John M. Dlugosz (Monsignor) on Sep 18, 2001 at 00:52 UTC
In general, it's intractable. There is no standard way to get an instantatnious snapshot, so you can't tell if you missed anything. Without knowing what was renamed to what, you don't know if a new name is one you already scanned or one you missed before. That could be helped if you had some way to uniquely identify the directory other than its name. Say, some key file that never changes and never gets renamed itself. You could have all the renamings and other changes go though an API that has hooks, or puts out a journal report, that your tools can use to know what's happening. Some OS's will do that for you directly. Some OS's have hooks or 3rd party tools to obtain a virtual "snapshot", meant for making consistant backups. —John	[reply]
Re: How do I count all the files in all subdirectories? by blakem (Monsignor) on Sep 17, 2001 at 23:10 UTC
Take a look at File::Find or find2perl. They will help you do just about anything you need to when it comes to walking directory trees.	[reply]
Re: Re: How do I count all the files in all subdirectories? by John M. Dlugosz (Monsignor) on Sep 18, 2001 at 00:46 UTC
I think File::Find will have the same problem.	[reply]