The problem is a classic use cases for counting with hashes, which others demonstrated very well.

Delegating it to sort - be it the shell command or the Perl built-in - is a waste of resources, which doesn't scale well.

Counting is O(n) but sort at best O(n log(n)) plus you suggest doubling the needed disc space, involving costly disc access time.

The time needed for a) merging and b) reading again and c) writing the sorted file can easily take more time than just reading once and incrementing a hash.

Additionally you are not saving much code, because the OP wants to chose the files dynamically.

If this was a one-shot problem your approach could be OK'ish, but I doubt that's the case in Bio Inf.

Last but not least

If it's possible to use sort, why not just using wc (word count) ?

Cheers Rolf
(addicted to the Perl Programming Language and ☆☆☆☆ :)
Wikisyntax for the Monastery

In reply to Re^2: Out of Memory when generating large matrix by LanX
in thread Out of Memory when generating large matrix by cathyyihao

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.