First off, your code doesn't work. $DIFF_OLD = <DIFF_OLD>; reads the first line of the file, not the whole file. If you want to slurp it, set $/ to undef.

Second, you're on Unix (at least, I'm assuming that based on your directory separator). So, why aren't you using the some of the commandline utilities to help you out? For example, I'm assuming that the only thing that will change is the YYYY / NNNN, plus the _1_ / _2_. So, why not create a temporary file for each one, with the crud stripped out, then using sort and diff? You can using Unix's sort to sort based on various keys, split using your own delimiter. Even with the temp files, this shouldn't take more than a second per 20k rows, and probably much less than that.

Being right, does not endow the right to be rude; politeness costs nothing.
Being unknowing, is not the same as being stupid.
Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

In reply to Re: Array loop is very inefficient, 100% CPU usage by dragonchild
in thread Array loop is very inefficient, 100% CPU usage by waiterm

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.