pwhysall has asked for the wisdom of the Perl Monks concerning the following question:

I return with another question that those skilled in the art might kindly help me with...

I want to recursively scan all the directories and subdirectories from a given starting point. At some point in the future I'm going to do things to the security on all files in the directory - the reason I'm using Perl is that (a) DOS batch is rubbish and (b) CACLS.EXE is rubbish and (C) I need to look at the security before doing the aforementioned things.

But enough natter; here's the subroutine that, at the moment, is intended to print out a recursive listing of the directory tree:

sub printdir { my $item; foreach $item(@_) { if (-d $item) { print "$item\n"; opendir(SUBDIR, $item) or croak "Can't open directory :$!"; my @subdir_items = readdir(SUBDIR); #+ closedir(SUBDIR); #+ printdir(@subdir_items); #+ } } }
At the moment, it simply prints out the very first eligible item, lots of times, with the odd warning about deep recursion.

This nicely printed out the items in a single directory before I added the lines marked with #+.

Replies are listed 'Best First'.
Re: Recursive directory scanning
by Tyke (Pilgrim) on Mar 01, 2001 at 19:08 UTC
    If you're doing *any* recursive directory stuff, use the File::Find module. It simplifies the above to
    perl -Mstrict -MFile::Find -wle 'find( sub {print}, @ARGV)' .
    
Re: Recursive directory scanning
by arturo (Vicar) on Mar 01, 2001 at 19:08 UTC

    Any sentence with "recursive subdirectory searching" on this site will be immediately followed by a mention of File::Find (link to CPAN) (don't worry -- it's a standard module!)

    You should, in all likelihood, be using it here. If you want examples, there are plenty on this site that should suffice, here's a handy link File::Find (link to PM Update which does nothing terribly useful but point to the documentation. Whoever suggested using Super Search was right)

    Philosophy can be made out of anything. Or less -- Jerry A. Fodor

Re: Recursive directory scanning
by Corion (Patriarch) on Mar 01, 2001 at 19:10 UTC

    What you really want to do is to use File::Find. There are a lot of threads here, look with the Super Search functionality (try "find files").

    There's nothing bad about File::Find - here's a short example that prints every filename (twice) :

    #!/bin/perl -w use strict; use File::Find; my @directories = ('.'); find( sub { # This sub gets passed the current filename relative to # the directory file::find recurses into - this is # weird and leads to many a confusing error. # Instead, I use $File::Find::name, which always gives # the full path (relative to the directory you were # in when you started File::Find). my( $filename ) = @_; print $File::Find::name, "\n"; print $filename, "\n"; }, @directories );

      The monks come up with the goods. Deepest gratitude to you all.

      Actually, I never though of this as a file-finding problem. I've obviously got my Win32 head on today, or else find(1) would have leapt immediately to mind.

      Out of sheer curiosity, though, I'd be grateful if someone could tell me what's wrong with my recursive function.

        I don't see any subdirectory building anywhere. When you ask for the directories under "/home", you're getting back "merlyn" and "vroom", not "/home/merlyn" and "/home/vroom". So you then recurse on just "merlyn", and the opendir fails.

        To do that portably, you need to look at File::Spec, since Unix wants forward slash, Windows can handle forward slash but most people expect backslash, Macs want colon, and VMS wants some mixed up mess.

        But then you also have to figure out how to skip over symlinks, since that can send you into an infinite loop by turning a DAG into a general node map, bringing you back to where you started too fast.

        You also didn't localize your directory handle. In the way you used it, it wouldn't have hurt you, but it it could have potentially stomped on any other use of the same name.

        So this is why "recursive directory handling" is nearly always responded with a groan, a sigh, and "please use File::Find". It's an easy task to consider, but a difficult task to actually do right.

        -- Randal L. Schwartz, Perl hacker

        I suspect the reason for the deep recursion is that you don't filter out '.' and '..'. When you recurse on the contents of a directory, you recurse on the contents of its "subdirectory" '.' (and its "subdirectory" '.' ...).

        As merlyn also pointed out, since you're not using chdir, you need to specify the full path to each file in the file test and the opendir().

        You already have the solution, which is File::Find. :)

        You're looping on the first item in the list which will be a '.'

        Works better if you add next if $item eq '.' or $item eq '..'; after the foreach

        readdir gives you the names in the directory... not path names, so when you call the printdir recursively you'll need to prefix the current directory to each element of the list. (Naturally you'll need to change your '.' test accordingly.

        Anyway this seems to work on my machine.

        #!perl -w
        use strict;
        use Carp;
        
        printdir(@ARGV);
        
        sub printdir {
            my $item;
            foreach $item(@_) {
              next if $item =~ /\.{1,2}$/;
              if (-d $item) {
                print "$item\n";
                opendir(SUBDIR, $item) or croak "Can't open directory :$!";
                my @subdir_items = readdir(SUBDIR);      #+
                closedir(SUBDIR);                        #+  
                printdir(map {"$item/$_"}@subdir_items); #+
              }
            }
          }
        

        Oh, by the way the equivalent in my previous post should have been

        perl -Mstrict -MFile::Find -wle 'find sub{print $File::Find::dir if -d},@ARGV' .
        
        but you saw that already, didn't you :)

        Update: Just saw merlyn's post. The above ran on Win32 because (as he says) the Windows port recognizes forward slashes as valid directory separators. I don't know whether this is an issue on other non unix ports. Still, I agree with him, it's a lot safer here to stay with the standard.

        Update 2: The '.' test is broken: it will match any file ending with a '.'. You'd need to split off the file name from the path, so you might as well use File::Spec... Oh good grief, why did I ever post this code? This'll teach me to shut up :(

A further thought, and a correction.
by pwhysall (Acolyte) on Mar 01, 2001 at 19:03 UTC
    The correction is that I did of course, also add the opendir(SUBDIR... line as well as the other 3 lines.

    I had assumed that each time I called printdir() a new SUBDIR filehandle, scoped to the instance of printdir() that I had just created, would spring forth. I suspect that that assumption is incorrect.

A reply falls below the community's threshold of quality. You may see it by logging in.