in reply to open, file handles and memory

while (<FILE>) is fine, just make sure it's not for (<FILE>) which _would_ load the file into memory, but otherwise seem mostly identical to while.

It's unlikly to be significant for parsing logfiles, but sometimes it's not optimal to go line by line, since this increases i/o actions, which in general are slow. On the other hand slurping the WHOLE file into memory will require lots of ram. You can strike a compromise by using reads of maybe half a megabyte (I don't know whats optimal, but that seems sensiblish to me :) you can process the file in larger chunks with more infrequent disk access, so a good change of a faster runtime. However this will make it marginally more complicated to write.

Replies are listed 'Best First'.
Re (tilly) 2: open, file handles and memory
by tilly (Archbishop) on Jan 13, 2001 at 19:51 UTC
[reply]
Re:ading lines
by orkysoft (Friar) on Jan 13, 2001 at 19:07 UTC
    When you instruct the Perl interpreter to read the next line of the file, it can't know in advance how long the line will be, so it's logical that it itself reads larger chunks at a time, and buffers the input, just like the OS does (YMMV).
[reply]
      Performing an 'strace' on this code: 
      open(F, "<README");   # arbitrary
$|=1;

while(<F>) {
    print "line $.\n";
}

      [download]
      Results in this: 
      open("README", O_RDONLY|O_LARGEFILE)    = 3
read(3, "\t\t    GNU GENERAL PUBLIC LICENSE"..., 4096) = 4096   # firs
+t block
write(1, "line 1\n", 7line 1
write(1, "line 2\n", 7line 2
...
write(1, "line 80\n", 8line 80
read(3, " and appropriately publish on ea"..., 4096) = 4096     # seco
+nd block
write(1, "line 81\n", 8line 81

      [download]
      So it does appear to buffer the data in chunks, but they seem to be managably sized. This too may differ depending upon OS or build of Perl.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom