Processing all files in a directory

billie_t has asked for the wisdom of the Perl Monks concerning the following question:

Yes, I realise this is getting to be a bit of a chestnut, but I'm having newbie problems opening each file in a directory, parsing each line.

my $dir = "e:\\work\\testlogs";

opendir DIR, $dir or die "opendir: $!";

foreach my $name (readdir DIR) {
        next if /^\./;
        print "$name \n";
           process_file ($name);
}
closedir DIR;

sub process_file {
    my $file = @_;
    my @date = split(/-/, $file);
    print "@date \n";
    
    open (FILE, $file) or die "Could not open file: $!";
    while (defined(my $line = <FILE>)) {      
          my @fields = split(/\s+/, $line);
          my $machine = $fields[0];
          close FILE;
      }
}
[download]

At the "foreach", I want each file in the directory to be opened, skipping files beginning with a ".". This is not working, with an error saying "use of unintialized value in pattern match".

More importantly, none of the files in the directory are being opened, with a "no such file or directory" being returned. Printing "$name", which I would hope contained the currently open file name, seems to produce ".", which is obviously what I'm trying to skip.

All wisdom much appreciated!

Comment on Processing all files in a directory Download Code

Replies are listed 'Best First'.
Re: Processing all files in a directory by sauoq (Abbot) on May 26, 2003 at 04:59 UTC
You have a couple of problems. `foreach my $name (readdir DIR) { next if /^\./;` [download] You want `next if $name =~ /^\./;` instead. The match operator works on `$_` by default. `sub process_file { my $file = @_;` [download] That doesn't do what you expect; it assigns the number of elements in `@_` to `$file`. You want either `my ($file) = @_;` or `my $file = shift;` or maybe even `my $file = $_[0];` (though I don't recommend that last.) `open (FILE, $file) or die "Could not open file: $!";` [download] Unless your CWD is `$dir`, you'll need to prepend it to your filename... otherwise it will try to open `$file` in the current directory. Oh, you might very well want `split ' ', $line` instead. That's a special case. Use `/\s+/` only if you want to generate a leading null field on lines that begin with space. Addendum: There are very good (security) reasons to use the three argument form of `open()` or to, at least, explicitly specify the mode. -sauoq "My two cents aren't worth a dime.";	[reply] [d/l] [select]
Re: Re: Processing all files in a directory by billie_t (Sexton) on May 26, 2003 at 05:32 UTC
Thanks sauoq, for all those points. Silly of me not to match to the correct variable. And I'm not speaking the perl lingo fluently enough yet to remember that dumping an array to a scalar just gives the number of items in the array!! Prepending the filename was what I needed as well - I assumed that something would "remember" from the opendir. I managed to fix the mis-positioning of the close file myself (I was only reading the first line of each file for a while there).	[reply]
Re: Processing all files in a directory by Abigail-II (Bishop) on May 26, 2003 at 12:26 UTC
Prepending the filename was what I needed as well - I assumed that something would "remember" from the opendir. Well, I can imagine people assume the results from `readdir` have the directory name prepended. But if you think so, you wouldn't use `/^\./` to weed out filenames, would you? Because then you would either select all filenames in the directory, or none at all, depending whether the directory name starts with a dot or not. Abigail	[reply] [d/l] [select]
Re: Re: Processing all files in a directory by billie_t (Sexton) on May 27, 2003 at 01:30 UTC
Re: Processing all files in a directory by Zaxo (Archbishop) on May 26, 2003 at 04:56 UTC
~~readdir does not return a list of all files. Each time it is called it returns the next name. That means you want a while loop for that.~~ To get a list of file names, call glob. It satisfies the need to skip dotfiles, too (Added) Oops, sauoq is correct - going back to see what is really failing. - Ah - `my $file = @_;` in the subroutine. you probably mean `my ($file) = @_;` The former puts @_ in scalar context, making $file equal the number of arguments given. After Compline, Zaxo	[reply] [d/l] [select]
Re: Re: Processing all files in a directory by sauoq (Abbot) on May 26, 2003 at 05:14 UTC
readdir does not return a list of all files. It most certainly does, when called in list context. -sauoq "My two cents aren't worth a dime.";	[reply]
Re: Processing all files in a directory by vek (Prior) on May 27, 2003 at 00:04 UTC
Do you want to skip files beginning with a . or do you just want to skip the . and .. directories? If it's the latter you can use the `-d` file test: `opendir (DIR, $dir) \|\| die "opendir: $dir - $!\n"; foreach my $name (readdir DIR) { my $fullPath = $dir . "/" . $name; next if (-d $fullPath); }` [download] -- vek --	[reply] [d/l] [select]