in reply to fast+generic interface for range-based indexed retrieval

How is your range of indexed keys defined? Can it be reduced to a sequence (numeric or alphabetical).

SQL could then do the trick: 

SELECT * FROM TABLE WHERE KEY_FIELD >= 'sequence_begin' AND KEY_FIELD 
+<= 'sequence_end'

[download]
or if the number of indexed keys is not too big: 
SELECT * FROM TABLE WHERE KEY_FIELD IN (key1, key2, key3, ... , keyN)

[download]

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Replies are listed 'Best First'.
Re^2: fast+generic interface for range-based indexed retrieval
by duckyd (Hermit) on Dec 10, 2008 at 23:22 UTC
    And if SQL can do the trick, but you want to avoid a full DBMS, you might look at DBD::SQLite
[reply]
      A valid comment.

      However, as the OP was speaking of hundreds of millions of records, I wonder whether SQLite would scale well for such large databases? I have only used it for small projects, but if it can handle in an efficient way such large volumes, I might consider it in the future. Do you have any experience in that respect?

      CountZero

      A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

[reply]
[reply]

In Section Seekers of Perl Wisdom