Oh great Monks, I'm coming for your advice.

I have written a perl script that counts the number of hits of specific numbers into user-defined bins. For example, this is my data file:

12
14
15
20
21

And I want to know how many hits I have in the following ranges:

1-19
20-29
30-39

So results would be like

1-19 3
20-29 2
30-39 0

I have done such a thing by fist saving my data into a hash (datahash), then saving my ranges into another hash (rangehash), and then basically going over all the data points in datahash and checking that the value falls within the ranges of the rangehash.

The problem is that for each datapoint in datahash, I loop through all the rangehash values and exit once I find the range where the datapoint falls. This is good for few data points, but now I'm having files with at least 2 million datapoints and 50,000 ranges, so looping through all of that just takes forever.

I was wondering if anyone would have better (more optimal) solution rather than just looping through the whole thing. Suggestions for other languages are well received!!!

Best,

Sakti

In reply to Perl hit counter for multiple ranges, optimization or options in other languages? by SaktiMonk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.