in reply to Readdir against large number of files

I'd try to use glob in scalar context: 

while (my $matchingFiles = glob("*.rtsd001")) {
   ...
}

[download]

Though that doesn't seem to your real problem - in your current version you already have only one filename in memory each time.

I'd rather think your problem is @fileContents, which can hold the contents of an entire file. Instead of reading a file, selecting lines, storing those, and printing the result, you can just print them line by line: 

               while (<UNMODIFIED>) {
                    /STRUC20/ and print MODIFIED $_;
               }

[download]
Perl 6 - links to (nearly) everything that is Perl 6.

Replies are listed 'Best First'.
Re^2: Readdir against large number of files
by learningperl01 (Beadle) on Oct 28, 2009 at 17:58 UTC
    Thanks for the help. I gave your code a try but it is only printing the line STRUC20. What I was looking to do is print all lines in the file after STRUC20. And also all files online contain the line STRUC20 once and they will all contain that line.
    Thanks again for all the help.

    Example file contents of file1.rtsd001 
    date time
blah blah
test test
STRUC20  #Code should look for this line and print everything after it
+.
need lines
need lines
print these lines

    [download]
[reply]
[d/l]
      What I was looking to do is print all lines in the file after STRUC20

      The flip-flop operator (aka the range operator .. in scalar context) might come in handy for that: 

      while (<UNMODIFIED>) {
    print MODIFIED $_ unless 1 .. /STRUC20/;
    # i.e. do print, except for first line upto a line that matches ST
+RUC20
}

      [download]
[reply]
[d/l]
[select]

      You're right, I didn't read your original code carefully enough.

      You can introduce a variable that stores if it's after the STRUC20 line: 

      
while (my $matchingFiles = glob("*.rtsd001")) {
    ...
    my $seen_struct;
    while (<UNMODIFIED>) {
        if ($seen_struct || /STRUCT20/) {
            $seen_struct = 1;
            print MODIFIED $_;
        }
    }
    ...
}

      [download]

      That way you don't have to store all the rest of the lines in memory, and still get the same semantics.

      Update: actually this solution looks more elegant, but does nearly the same thing.

      Perl 6 - links to (nearly) everything that is Perl 6.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom