If you do it like: 

perl -ne'print $_ if ! /^<es\s*$/' huge.xml | perl extract.pl -

[download]

Then you don't even have to wait for the huge file to be read twice. The time required could well be almost the same as it would be without the filter. Since the filter code likely can run faster than the XML parsing code, the difference in run-time could just be the insignificant time it takes to filter one buffer's worth of XML. It would likely take a bit more CPU (probably less than 2x), but I doubt processing a huge XML file is usually CPU-bound on most systems.

Though, brettski didn't seem to find even that proposal acceptable when I proposed it in chat around the time that the root node was posted.

- tye        

In reply to Re^3: Ignoring not well-formed (invalid token) errors (pipeline) by tye
in thread Ignoring not well-formed (invalid token) errors by brettski

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.