Use 'ls' to read files in perl????

ad23 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I am trying to read multiple files in my perl program. I am using UNIX "ls" command to do the task.

I need to know whether it is the best way to do this to read from following directory structure:

/Data_Files/2010_07_07_15h32m/Data.txt
/Data_Files/2010_07_07_16h21m/Data.txt
/Data_Files/2010_07_07_16h24m/Data.txt
/Data_Files/2010_07_07_18h38m/Data.txt
/Data_Files/2010_07_07_19h11m/Data.txt
[download]

#!/usr/bin/perl 
#my @files = ('data1.txt', 'data2.txt', 'data3.txt' );
my @files = `ls Data_files/*/Data.txt`;

foreach my $filename(@files){
    open FILE, $filename or die "Cannot open $filename for reading: $!
+\n";   
    ### Read Line by Line ###
    
    while($line=<FILE>){
    #read file line by line
    }
close(FILE);
}
[download]

Thanks!!!

Comment on Use 'ls' to read files in perl???? Select or Download Code

Replies are listed 'Best First'.
Re: Use 'ls' to read files in perl???? by runrig (Abbot) on Jul 07, 2010 at 19:00 UTC
No need for ls, use glob: `my @files = glob('/Data_Files/*/Data.txt');` [download]	[reply] [d/l]
Re: Use 'ls' to read files in perl???? by FunkyMonk (Bishop) on Jul 07, 2010 at 20:47 UTC
runrig and AndyZaft have given you better ways of doing what you want. I'd just like to tell you why your method doesn't work. Backticks capture the output of an external command. In list context (as you have) they return a list of lines, and each of those lines ends in a linefeed (on NIX platforms). You need to remove them using chomp: my @files = `ls Data_files//Data.txt`; chomp @files; [download] or `foreach my $filename(@files){ chomp $filename;` [download]	[reply] [d/l] [select]
Re^2: Use 'ls' to read files in perl???? by ad23 (Acolyte) on Jul 07, 2010 at 21:08 UTC
Thank you all!!!	[reply]
Re: Use 'ls' to read files in perl???? by AndyZaft (Hermit) on Jul 07, 2010 at 19:31 UTC
I always used this form: `my @files = <Data_files/*/Data.txt>;` [download]	[reply] [d/l]
Re^2: Use 'ls' to read files in perl???? by eyepopslikeamosquito (Archbishop) on Jul 08, 2010 at 08:07 UTC
Perl Best Practices, Chapter 8, "Built-in Functions", page 167: "Use glob, not <...>". Conway argues that `<>` is overloaded with too many different meanings in Perl, leading to maintenance problems. After noting that changing: `my @files = <.pl>;` [download] to: `Readonly my $FILE_PATTERN => '.pl'; my @files = <$FILE_PATTERN>; # KABOOM! (probably)` [download] blows up in your face, he concludes that "a construct that breaks when you attempt to improve its readability is, by definition, unmaintainable". Use `glob` and keep the angle brackets strictly for input operations. Update: Further to PBP, note Larry's sensible advice on language design: Common operations should be "Huffman coded". That is, frequently used operators should be shorter than infrequently used ones. In this case, file input is a more common operation than file globbing and so deserves the `<...>` shortcut. This was proposed back in 2000 in rfc 34: Angle brackets have two separate meanings in Perl - file globbing and line input from a filehandle. This means that perl attempts to Do What I Mean and often misinterprets me. Since file globbing can be accomplished with the glob function and since file input is the more common use of angle brackets, they should not be used for file globbing. Curiously, in Perl 6, `<...>` is no longer used for file input or file globbing, but as an improved version of Perl 5's `qw` operator. For file input, Perl 6 uses either `$fh.readline()`, or the shorter form `=$fh`. The Perl 5 built-in glob function is no longer built-in in Perl 6, but is now part of the IO modules (according to S29).	[reply] [d/l] [select]
Re^3: Use 'ls' to read files in perl???? by AndyZaft (Hermit) on Jul 08, 2010 at 13:43 UTC
Yup, I figured glob is better, although I never read the PBP stuff before. Thank you for pointing it out and for the links, I like to learn how to use tools better. Even if only use them occasionally, it looks like I only use perl every decade or so. In any case I guess it was good to give him a different solution because this way we got an explanation why he should stick to glob :)	[reply]
Re: Use 'ls' to read files in perl???? by cdarke (Prior) on Jul 08, 2010 at 07:04 UTC
If, in a moment of temporary weakness, you are tempted to use a UNIX utility program from Perl again, you might like to check UNIX 'command' equivalents in Perl. It is not complete, but its a start.	[reply]
Re: Use 'ls' to read files in perl???? by perreal (Monk) on Jul 08, 2010 at 08:02 UTC
There is also the forking factor, when ls is used a new process is created with all the overhead. There is also the portability factor, when there is no ls command in the os.	[reply]
Re^2: Use 'ls' to read files in perl???? by JavaFan (Canon) on Jul 08, 2010 at 10:06 UTC
There is also the forking factor, when ls is used a new process is created with all the overhead. Yes, forking a new process totally dwarves going to disk and searching the filesystem. There is also the portability factor, when there is no ls command in the os. Yeah, but I'd figure the set of OSses that don't come with an 'ls' is about the same set of OSses that don't come with `perl`. And I wouldn't assume the OP has the intention to take his UNIX centric approach and run it in a non-UNIX environment.	[reply] [d/l]