Thanks for all the responses. In fact I am using BerkeleyDB's BTrees (and cursors) to get good performance, it's just that my code winds up being specialized for each database and rather ugly. My keys (to date) are usually floating point numbers which I format like %012.7f since these numbers will never exceed 9999. I perform the one-time database load using db_load.

It sounds as though I can't do much better than my current solution and still obtain good performance. One thing that I can do to clean things up is use a Perl iterator.

Perhaps I can set this up to include a few parameters and hooks to define the format of the query string. This would yield the generic interface that I'm looking for, and perhaps would be worth releasing as a module.

The Bowtie remark was interesting; I'm well-acquainted with that relatively obscure package (as well as the equally obscure Dynamite), but don't think that the very cool and underutilized Burrows-Wheeler transform is applicable here.

Thanks again to all ...

In reply to Re^2: fast+generic interface for range-based indexed retrieval by jae_63
in thread fast+generic interface for range-based indexed retrieval by jae_63

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.