ybiC has asked for the wisdom of the Perl Monks concerning the following question:

The following scriptlet reads a list of directories for files in them, then "does stuff" with each file.   The "if (-d $_)" loop is *supposed* to detect subdirectories and add them back to the list of dirs to read.   In other words, drill down the directory structure and process every file in every subdirectory.

But it only processes the listed directories, not child subdirectories.

I thought I understood perlfunc:_X, push, and next.   What simple silly syntax am I doing wrong?   Perl 5.00503 btw.
    cheers,
    Don

#!/usr/bin/perl -w use strict; my @dirs = ( # omit trailing slash '/var/www', '/home/me', '/usr/games', ); for my $dir(@dirs) { opendir DIR, $dir or warn "Error opening $dir:\n$!"; my @infiles = (readdir DIR) or warn "Error reading $dir:\n$!"; closedir DIR or warn "Error closing $dir:\n$!"; for(@infiles) { $_ =~ s/^(\.|\.\.)$//; if (-d $_) { push @dirs, $_; next; } # do stuff } } for(@dirs) { print " $_\n"; }

Replies are listed 'Best First'.
Re: Read directories' contents, add subdirs back into directory list
by Vynce (Friar) on Jun 02, 2001 at 10:52 UTC

    while i agree that it is probably a good idea to use someone else's wheel, it is often helpful to figure out what's wrong with the round thing you thought you were building. since no-one else spoke of your spokes, i stuck a stick in them myself, and i think i have it.

    i think most of your problem is in this section:

    a: for(@infiles) { b: $_ =~ s/^(\.|\.\.)$//; c: if (-d $_) {
    at line a, you are implicitly setting $_ to items from the directory $dir. in line b, you throw out . and .., and in line c you test to see if $_ is a directory. however, $_ is, in all these cases, relative to $dir -- and perl has no way of knowing that. -d needs the full path or, i believe, the path relative to the location of the script that's running. so here's a quick re-write that will fix this:

    #!/usr/bin/perl -w use strict; my @dirs = ( # omit trailing slash '/var/www', '/home/me', '/usr/games', ); for my $dir (@dirs) { opendir DIR, $dir or warn "Error opening $dir:\n$!"; my @infiles = (readdir DIR) or warn "Error reading $dir:\n$!"; closedir DIR or warn "Error closing $dir:\n$!"; for(@infiles) { $_ =~ m/^\.{1,2}$/ and next; my $d = "$dir/$_"; if (-d $d) { push @dirs, $d; next; } # do stuff } } for(@dirs) { print " $_\n"; }

    some notes:

    first, though this treatment of the array works, i don't like it. manipulating the array you're looping on is vaguely dangerous, just because you get tempted to do much worse things than push onto the end of it. i'd replace that with a while ($dir = shift @dirs), and push $dir onto @keep if you need to keep them for later.

    second, you'll note i changed your regex line. i made two changes, both possibly minor stylistic things. one was use of the {1,2} to indicate "one or two" because i find that easier to read than the multiple backslashes and alternation. two, rather than turning those into blank entries and keeping them, if it matched that, i next. why? a little lower, i generate an absolute path by appending the local dirname ($_) onto my current "base" path ($dir), with a slash. if i kept blank ones, this would add the base path back in twice -- infintitely filling the array. ew. bad.

    i'd also recommend not using $_; you're naming it explicitly anyway, you might as well give it a real name. but that's totally my own bias, so ignore that.

    .

    anyway, the moral of the story is "use those other modules" -- but the other lesson is "perl can't guess what you're thinking more than about 90% of the time."

    .
Re: Read directories' contents, add subdirs back into directory list
by bikeNomad (Priest) on Jun 02, 2001 at 02:26 UTC
    Well, for one thing, you're modifying @dirs while iterating through it. I'm not sure how safe that is (though I'm not sure how unsafe it is, either).

    More importantly, how deep are you trying to go? Even if you use another collection, the names you get from readdir are relative to their parent. So a readdir on them isn't going to yield any more subdirs. If you really want to recurse, you're going to have to do something different.

    Perhaps File::Find would do the trick for you:

    use strict; use File::Find; my @dirs = ( '/home/ned', '/usr/games' ); sub doStuff { print shift() . "\n"; } find( sub { doStuff($File::Find::name) unless -d _ }, @dirs );
Re: Read directories' contents, add subdirs back into directory list
by mlong (Sexton) on Jun 02, 2001 at 02:25 UTC
    Use Recursion. I built the following script to recurse through my mp3 directories to build a playlist. I use this on Win32 primarily, but it works on linux/unix too. Just specify the separator on the command line when you run it.

    #!/usr/bin/perl use MP3::Info; $dHandle = 'dh00'; my $startDir = shift || '.'; my $separator = shift || '\\'; parseDir($startDir); sub parseDir { my $currentDir = shift || return; my $dirHandle = $dHandle++; opendir($dirHandle, $currentDir) or die "Coudln't open directory $ +startDir.\n"; while (my $filename = readdir ($dirHandle)) { next if ($filename =~ /^\./); # go to next filename if we have + a special file (i.e. . or ..) # if we find a directory, parse it by calling the pars +eDir again ( Recursion ). if( -d "$currentDir$separator$filename" ) { parseDir("$currentDir$separator$filename"); } elsif($filename =~ /mp3/i) { my $mp3 = new MP3::Info "$currentDir$separator$filename"; unless ($mp3 == undef) { print "$currentDir$separator$filename\n"; } } } closedir($dirHandle); } exit(0);


    Notice the line: $dHandle = 'dh00'; This variable gets incremented and the subsequent value value gets used as the directory handle in the current iteration.

    Let me know if you have more questions about this.

    -Matt

    Edit by tye

(zdog) Re: Read directories' contents, add subdirs back into directory list
by zdog (Priest) on Jun 02, 2001 at 04:06 UTC
    You may also want to take a peek at File::Recurse which recurses through directory structures for you.

    Zenon Zabinski | zdog | zdog7@hotmail.com

Re: Read directories' contents, add subdirs back into directory list
by Beatnik (Parson) on Jun 02, 2001 at 12:58 UTC
    This worked for me in the past, altho I stick to File::Find these days :)
    sub dodir { opendir(DIR,$_[0]); my $dir = $_[0]; if ($dir !~ /\/$/) { $dir .= "/"; } my @List=readdir(DIR); closedir(DIR); splice(@List,0,2); print "Reading $_[0]\n"; foreach $file (@List) { my $file = $dir.$file; if (-d $file) { dodir($file); } else { push(@Files,$file); } } } dodir($ARGV[0]); foreach(@Files) { print $_,"\n"; }
    Greetz
    Beatnik
    ... Quidquid perl dictum sit, altum viditur.
Re: Read directories' contents, add subdirs back into directory list
by clintp (Curate) on Jun 02, 2001 at 17:43 UTC
    In general, if you're using for() to iterate over a list that you're modifying within the loop: that's a danger sign. Something will get messed up sooner or later, and be tough to track down. Instead, try this simple structure instead of the for:
    while(@dirs) { my $dir=shift @dirs; # Rest of loop here }
    This way you can throw stuff anywhere you want to in @dirs, and it'll all get processed eventually. By changing the push to an unshift, you'll get a depth-first search instead of breadth-first. Nifty, eh?

    You do have a couple of other small bugs, which other posters have pointed out.