During the course of my day i had to parse and process at 2m+ line text file. No worries i thought: 
open(FH, 'someLargeFile.txt') || die "Unable to open file: $!";
foreach (<FH>) {
    ..
    #do something
}

[download]

I thought this would loop thru each line of the file and #do something. I was wrong, and of course i felt like a bit of a n00b.

What i saw happening on my little sparc was the entire file being loaded into memory, then being processed, and because it was so large, it didnt fit, and abended.

So being an "experienced n00b" i tried a couple of things and found out that if i s/foreach/while/ it seems to iterate over everything without loading it all into memory 1st.

When i think about it, it kinda makes sense, in that how can foreach know when to stop, when it doesnt know where the end is, and while its just going until it finds an EOF marker.

Moral of the story, if something is not working as you expected, its probably not designed to, _or_ you can think you've got experience in things, you probably have, but the basics can still jump up and bite.

n00bily yours...

Update: as pointed out by gmax jargon can sometimes be difficult. in this case n00b == newbie == someone new to doing something.

In reply to file processing, while and foreach: a n00b experience by Ryszard

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.