Recursive file search

mellin has asked for the wisdom of the Perl Monks concerning the following question:

I have a need for a function in my program that would return back all xml-files it can find. The function needs to be recursive, because it goes through directory like this:

archive
.2005
..12
...420.xml
..11
...419.xml
..10
...418.xml
...417.xml
.2004
..9
...416.xml
..7
...415.xml
.2003
..5
...414.xml
..4
...413.xml
..2
...412.xml
[download]

So directory 'archive' has subdirs based on years and they have subdirs based on months, latter ones then have these xml-files that i would like to read on to array. When the funtion has read recursively all subdirs and pushed xml-files to array, it then returns that array. Where i'm facing problem, is with this code that i've copied from web and modified to suit my needs:

#!/usr/bin/perl

my @files;
loopDir('./');
print "found xml-files $#files..\n";

sub loopDir {

local ($dir) = @_;
chdir $dir or die "Cannot chdir to $dir\n";
local(*DIR);
opendir DIR, '.';

while ($f=readdir(DIR)) {

        next if $f =~ /^\./;
        push @files, $dir . '/' . $f if $f =~ /.*\.xml$/;

        if (-d $f) {
                &loopDir($f);
        }
}
closedir(DIR);
chdir("..");
[download]

There doesn't seem to be any other way to introduce that '@files' array, other than higher in the program. I would like this loopDir function to return it, not modify elsewhere introduced array. Like this i mean: my @files = loopDir('./');

Comment on Recursive file search Select or Download Code

Replies are listed 'Best First'.
Re: Recursive file search by Corion (Patriarch) on Feb 20, 2005 at 01:04 UTC
Perl has a module for finding files, called File::Find, and there are also File::Find::Rule, which even has an XML-file-searcher I think, and merlyns File::Finder, which is similar yet different from File::Find. Your code with File::Find::Rule would look like this: `use strict; use File::Find::Rule; sub loopDir { File::Find::Rule->file->name(/\.xml$/i)->in(@_); };` [download] But maybe you want also to search within the XML files in one go, then File::Find::Rule::Wellformed and File::Find::Rule::XPath might become of interest.	[reply] [d/l]
Re: Recursive file search by blahblahblah (Priest) on Feb 20, 2005 at 03:51 UTC
File::Find and the other modules that Corion pointed out are, in my opinion, the easiest way to solve this problem. To answer your question about returning values from a recursive sub, since that could be generally useful for other things, you could do something like this: `sub loopDir { my @files; local ($dir) = @_; chdir $dir or die "Cannot chdir to $dir\n"; local(DIR); opendir DIR, '.'; while ($f=readdir(DIR)) { next if $f =~ /^\./; push @files, $dir . '/' . $f if $f =~ /.\.xml$/; if (-d $f) { # push the subdirectory's files onto @files push @files, &loopDir($f); } } closedir(DIR); chdir(".."); # return the list of files return @files; }` [download]	[reply] [d/l]
Re^2: Recursive file search by merlyn (Sage) on Feb 20, 2005 at 04:24 UTC
`if (-d $f) {` [download] This breaks horribly for symlinks, which is why you shouldn't code this yourself. -- Randal L. Schwartz, Perl hacker Be sure to read my standard disclaimer if this is a reply.	[reply] [d/l]
Re^3: Recursive file search by mellin (Scribe) on Feb 20, 2005 at 10:09 UTC
I'm not really sure wether i'm needing this feature or not. I know i should always prepare for the unexpected, but there should never be symlinks on these folders. I'll take the hint though and look for the direction of File::Find.	[reply]
Re: Recursive file search by Miguel (Friar) on Feb 20, 2005 at 15:34 UTC
Another alternative: `#!/usr/bin/perl -w use strict; use IO::All; my @files; for ( io('.')->All ) { push @files, $_ if $_ =~/.*\.xml$/i; }` [download] I spent 22 seconds to read 33721 files/directories on a PIV 1.7 with 256Mb RAM	[reply] [d/l]
Re: Recursive file search by sh1tn (Priest) on Feb 20, 2005 at 22:52 UTC
In addition to the excellent examples given: `# find "captured" in subroutine use strict; use File::Find; my @dirs = qw( /devel/dir1 /devel/dir2 ); my $exten = '.xml'; my @files = _get_files(\@dirs, $exten); sub _get_files{ my $dirs = shift; my $what = shift; my @files; my $want = sub { -f && /\Q$exten\E$/ && push @files, $File::Find::name }; find($want, @{$dirs}); @files }` [download]	[reply] [d/l]