Re: How to read in large files

You're reading the complete file into memory before doing anything. If your program logic allows for this, it's easy to rewrite it by processing the file line-by-line. Change:

foreach my $line (<$readHandle>) {
[download]

while (defined( my $line= <$readHandle>)) {
[download]

Comment on Re: How to read in large files Select or Download Code

Replies are listed 'Best First'.
Re^2: How to read in large files by kennethk (Abbot) on Jan 28, 2016 at 15:50 UTC
Corion has correctly identified the issue, but I think a little more explanation might be helpful. When you use `foreach`, perl constructs the entire list before iterating over it. Using `while`, on the other hand, executes exactly 1 read attempt per loop since `while` executes after each true evaluation (successful read). In addition, the stored value goes out of scope at the end of each iteration, thus you only store 1 line at a time instead of all of them. #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.	[reply] [d/l] [select]
Re^3: How to read in large files by davido (Cardinal) on Jan 28, 2016 at 16:42 UTC
And this nuance occurs because: In the case of `foreach ( EXPRESSION ) { BLOCK }`, the `EXPRESSION` is evaluated in list context. The `<FILEHANDLE>` operator returns a list of records from the file when evaluated in list context. Logical records are typically based on lines within the file. In the case of `while ( EXPRESSION ) { BLOCK }`, the `EXPRESSION` is evaluated in scalar context for its Boolean value. The `<FILEHANDLE>` (diamond) operator returns a single record from the file when evaluated in scalar context. Dave	[reply] [d/l] [select]
Re^4: How to read in large files by Only1KW (Sexton) on Jan 28, 2016 at 17:33 UTC
Thanks all. I was unaware foreach was different from while other than syntactically.	[reply]