use strict;

use File::Spec::Functions 'catfile';
use IO::Dir;

sub _dircnts {
  my $dir = shift;
  my $do  = IO::Dir->new($dir) or die "Couldn't open '$dir': $!";
  return
    map catfile($dir, $_),
    grep !/^\.\.?$/, $do->read;
}

sub rec_readdir {
  my $dir  = shift;
  my @cnts = _dircnts $dir;

  my $iter; $iter = sub {
    my $f = pop @cnts;
    return unless defined $f;

    push @cnts, _dircnts $f
      and return &$iter
        if -d $f;

    return $f;
  };

  return $iter;
}

my $di = rec_readdir(shift);
while(my $file = &$di) {
  # do stuff here
}
[download]

Although if you're actually iteratively walking a directory tree then File::Find::Rule will do a much more reliable job (as it in turn uses File::Find) and is oh-so useful :)

For Perl, it is very easy to convert recursive program into its iterator version.

The only thing you need, in order to accomplish this conversion easily, is 'stack'. Perl by default suppots stack in its core, thanks to the ability to pop() and push() against a list.

To convert:

• As a start point, some element(s) need to be processed is push() onto the stack
• For each iteration:
  • pop() one element out of the stack, if nothing to pop(), the process is finished; otherwise process this element
  • If there are child element(s) need to be processed, push() them onto the stack

{
    my @fq;   # files
    my @dq;   # directories
    my $currentDir;

    sub initDir { push @dq, shift; };
    
    sub getNextFile {
        my $f;
            
        if ( $#fq >= 0 ) {
            $f = pop @fq;
        } else {
            if ( $#dq >= 0 ) {
                my $r;       # filename read
                local *D;
    
                $currentDir = pop @dq;
                opendir(D, $currentDir) || die "$currentDir: $!";
                
                while ( defined( $r = readdir( D ) ) ) {
                    next if $r =~ /^\.{1,3}$/;
                    
                    my $ff = $currentDir . '/' . $r;
                    if ( -d $ff ) {
                        push @dq, $ff;
                    } else {
                        push @fq, $ff;
                    }        
                }
                
                closedir D;
                $f = getNextFile();
                
            } else {
                $f = undef;
            }
        }
        return $f;
    }
}

my $dir = "c:/inetpub";

# execution start here

initDir( $dir );
my $f;
my $i = 0 ;
while ( defined( $f = getNextFile() ) ) {
    $i++;
    print "[$i] $f \n";
}
[download]

To me, recursion means "let the programming language manage the stack", so the use of either an explicit stack or queue bothers me when I'm trying to do things recursive-like.

So here's a somewhat more fully iterator-style solution that uses the dirhandle as an iterator, and keeps a single scalar as the memory between invocations. Oh, and this isn't totally necessary, but I convert simple files to iterators that return the file once. I'm doing that because whenever I write iterators, I always use pretty much the same pattern: compute the next value if needed, consume a result from a sub-iterator, then return it. This allows a class hierarchy of iterators that override only the minimum necessary operations. Anyway, here's the code:

use IO::Dir;
sub make_reader2 {
    my ($dirname) = @_;
    my $d = new IO::Dir $dirname or die "opendir $dirname: $!";
    my $next;

    return sub {
        while (1) {
            unless (defined $next) {
                my $entry = $d->read or return;
                next if $entry eq '.';
                next if $entry eq '..';
                $entry = "$dirname/$entry";
                if (-d $entry) {
                    $next = make_reader2($entry);
                } else {
                    $next = sub { my $r = $entry; undef $entry; $r; };
                }
            }

            my $result = $next->();
            return $result if defined $result;
            undef $next;
        }
    };
}

my $reader = make_reader2($ARGV[0]);
print "$_\n" while (defined($_ = $reader->()));
[download]

What do you think about it? is there anything fundamentally flawed with this approach?

Uh, perhaps excessive complexity, when File::Find is better, faster, and cheaper?

-- Randal L. Schwartz, Perl hacker
Be sure to read my standard disclaimer if this is a reply.

Yes, of course; but this was just a way to play around with both ideas and learn something. I would never waste a couple of hours studying a problem that's already solved if it's not to learn something from it :-)

I wrote that once in Re: Re: (Perl6) Groking Continuations (iterators) as a general response to converting a recursive function to an iterator. When it is published, see also Dominus' Perl Advanced Techniques Handbook which goes into this subject in depth.