Depth Listing in Directory Traversal

crabbdean has asked for the wisdom of the Perl Monks concerning the following question:

For this I'll refer to a previous node 349582 and especially tye's response to that node 350948 in which he describes a "next" method iterator.

I've since used the guts of this "next" method (Big Thanks to tye) in a module I'm writing. I'm wanting to amend that method so that you can specify it to only go so deep into the directory tree. Here is the code:

sub this_deep {
    my ($self, $depth) = @_;
    $depth = -1 unless @_ > 1;
    $self->{depth} = $depth;
}

sub next {
    my $self = shift;
    while( 1 ) {
        if (  @{ $self->{output} }  ) {
            my $line = shift @{ $self->{output} };
            return $line;
        }
        if ( ! @{ $self->{dirs} } ) {
            return;
        }
        my $dir = shift @{ $self->{dirs} };
        if (-d $dir) {
            if(  opendir( DIR, $dir )  ) {
                map {
                    my $file = $_;
                    my $fullfile = File::Spec->catfile( $dir, $file );
                    if (-d $fullfile && ($self->{level} <= $self->{dep
+th} || $self->{depth} == -1)) {
                        push (@{ $self->{dirs} }, $fullfile);
                    }
                } File::Spec->no_upwards( readdir(DIR) );
                closedir DIR;
                
            } else {
                warn "\a\a\aopendir FAILED, $dir: $!";
            }
        } else {
            warn "\a\a\aNOT A DIRECTORY: $dir\n";
        }
    }
}
[download]

As you can see in this code I've put in the checks for the $self->{level} in that it will only push to the $self->{dirs} stack if the current level is below the specified depth or depth == -1 (if depth is set to -1 it will fully recurse the directory tree)

I'm having trouble building into this how to track the depth to ensure it only goes as far as specified. Any suggestions?

Update: I should add that what makes this tricky is the code makes it go DOWN a directory tree first before starting on the next level, not across the directory tree as is usually the case.

Dean
The Funkster of Mirth
Programming these days takes more than a lone avenger with a compiler. - sam
RFC1149: A Standard for the Transmission of IP Datagrams on Avian Carriers

Comment on Depth Listing in Directory Traversal Download Code

Replies are listed 'Best First'.
Re: Depth Listing in Directory Traversal (one more stack) by tye (Sage) on Jul 20, 2004 at 06:36 UTC
Where you `push @{ $self->{dirs} },...` you need to add `push @{ $self->{levels} }, 1+$self->{level};`. When you shift off of `$self->{dirs}`, also do `$self->{level} = shift @{ $self->{levels} };`. You can do it in more memory-efficient ways but the added complexity seems not worth it. - tye	[reply] [d/l] [select]
Re^2: Depth Listing in Directory Traversal (one more stack) by crabbdean (Pilgrim) on Jul 20, 2004 at 13:16 UTC
tye you are great. :-) Love your work! ++ That worked a treat. You know I was part way there in that thinking process, had similar thoughts but hadn't connected the pieces. Appreciate the help. Thanks also to hbo that idea would have worked also but I felt this was cleaner. Dean The Funkster of Mirth Programming these days takes more than a lone avenger with a compiler. - sam RFC1149: A Standard for the Transmission of IP Datagrams on Avian Carriers	[reply]
Re: Depth Listing in Directory Traversal by hbo (Monk) on Jul 20, 2004 at 04:36 UTC
`sub next { my $self = shift; while( 1 ) { if ( @{ $self->{output} } ) { my $line = shift @{ $self->{output} }; return $line; }` [download] Since we see no other references to this array ref, I'm assuming this code is cut down from its full version. `if ( ! @{ $self->{dirs} } ) { return; } my $dir = shift @{ $self->{dirs} }; if (-d $dir) {` [download] Add a proper declaration for `$olddir` above, and: `if ($oldir){ $self->{level} += ($dir=~tr!/!/!) - ($olddir=~tr!/!/!); } $olddir=$dir;` [download] I haven't tested this, but the idea is that, since the paths are all absolute, a change in level means a difference in the number of path seperators. (You might want to use a more portable form of the seperator than a literal "/".)	[reply] [d/l] [select]
Re^2: Depth Listing in Directory Traversal by crabbdean (Pilgrim) on Jul 20, 2004 at 04:56 UTC
Well I'm not sure this will work because the code makes the program go down the directory first, not across. Based on your idea it would have to keep a record somehow of the $oldir variable for when it finally finished going down the tree and came back to start on the next level. This requires it to keep a kind of memory of the tree and what it has or hasn't done. There must a simpler way. BUT ... your idea has merit and something I'd considered also - your post has made me rethink it. If the number of path seperators is greater than $depth then don't push to the stack. That may work. I just don't like using relying on counting "\" in strings as a reliable method of keeping track of depth. If by chance a directory has that character in it, the routine will fail. Dean The Funkster of Mirth Programming these days takes more than a lone avenger with a compiler. - sam RFC1149: A Standard for the Transmission of IP Datagrams on Avian Carriers	[reply]
Re^3: Depth Listing in Directory Traversal by hbo (Monk) on Jul 20, 2004 at 05:35 UTC
On Windows,"\" is the path seperator. A file name can't contain that character, so you won't encounter it unless it means a depth transition in the stack. Same goes for "/" on Unix. The tricky part with Windows would be a rooted path like "c:\\". For that you could count occurrences of "\+",although I believe that ActivePerl translates the seperators to "/" for portability with Unix. The "width first" traversal of the directory tree doesn't matter in the code I gave. The path seperator count will be the same between peer directories, thus their difference will be zero, and we will add zero to the depth counter. This also works for transitions up the stack, since that difference will be negative.	[reply]
Re: Depth Listing in Directory Traversal by adamk (Chaplain) on Jul 21, 2004 at 04:26 UTC
In general, when doing queue-based-recursion, just unshift onto the queue, rather than pushing. `my @queue = ( $start ) { while ( my $thing = shift @queue ) { do_something_with( $thing ); unshift @queue, $thing->children; }` [download] I'm assuming this is what you are after?	[reply] [d/l]