I'm trying to capture the output of an external program. for this, i've always used backticks...
`program args`

[download]
However i've run into the situation where the program i need to run for a particular script prints a tremendous amount of output. (I'm talking on the magnitude of hundreds of megabytes). I've never bothered myself with the details of backticking, however now its an issue. even in a
while(`program args`)

[download]
loop, perl aparently stores all of the data internally and waits for the program to finish before acutally starting the loop. This is unacceptable in this situation, as it will easily bring the system to its knees.

Instead, i used 
open PROG "program args |"
while (<PROG>) {
...
} close PROG

[download]
to aleviate this strain, but is this really the best way of doing it? Has anyone else run into this problem in the past, and how was it handled?

In reply to External program with large amounts of output by cyberconte

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.