I know this is a Perl forum, but sometimes Perl by itself is not the best tool for the job. You might consider testing the speed of using another program like sort(1) on the file before processing it with Perl (or just run sort from your Perl program).

Check out the helpful advice in this thread: Fast/Efficient Sort for Large Files

Then, you can read in the lines until you have the top N values and you're done.

If your log files are not easily sortable using sort(1) because of strange formatting and delimiters (like web server access logs), you may pre-process them (with Perl, of course) so that they have nice column separators for sort to recognize.

In reply to Re: Re: Re: *Fastest* way to print a hash sorted by value by esh
in thread *Fastest* way to print a hash sorted by value by smellysocks

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.