Corion and eyespoplikeamosquito have discussed your specific code examples, so I'll discuss the general question:

  1. When we should choose to slurp a file into array and processing it
  2. Processing it line by line
  3. What are the pro and cons of using either technique

The short answer is that you should always process files line by line unless you can't otherwise avoid it.

Times when you might not be able to avoid it are when you wish to do some sort of processing on the whole file, for example, sorting all the lines (although there's usually better tools for this) or picking out a random quote (although there are better tools for this too).

If you don't have to be able to access all the lines of a file in a semi-random order then you should always process it line by line.

Why? Because one day someone will feed your program a very large file. Perhaps one which is 10 Gb big. If you process that line by line your memory footprint will be comparatively small, if you slurp it all into memory, then your memory footprint will be *at least* as big as the file, sometimes almost twice that, and your machine might come grinding to a halt.

The pro of processing a file line by line is that your program stays fast and usable. The con is that you can only refer to lines you've already seen (if you've stored them) and the current lines. Lines further into the file are much more difficult to access. Fortunately you don't usually have to.

The pro of slurping a whole file into memory is that you can access any line of it easily. The con is that you might need a lot of memory, and it can really slow your machine down.

Hope that helps.

jarich

In reply to Re: Slurping file into array VS Process file line-by-line by jarich
in thread Slurping file into array VS Process file line-by-line by monkfan

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.