How many lines in your file?

18885025

Over what range are the keys and values spread?

keys from 6 to 164664533
values from 0 to 494514

How many lookups per load?

This is a stand-alone system application that does up to thousands lookups per run (although it could be called for just one lookup).This is an intermediate step in the app. The others step are well optimized for performance.

Is concurrency a requirement?

In principle, no

How often does the dataset change?

More or less, once in a month

How fast a lookup are you hoping for?

As much as I can. As I told before, the rest of the script is optimized for speed and memory usage and it is a bit frustrating to see how the others parts of the program (which seems to be much more complicated), runs very fast and these lookups are slowing down so much the overall process

Thanks,

citromatik

In reply to Re^2: fast lookups in files by citromatik
in thread fast lookups in files by citromatik

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.