in reply to Re^2: perl database question
in thread perl database question

how do hashes deal with files up to several GB in size
You will note this is the first time you mentioned gigs of data. What you had asked for was
a convenient way to store, retrieve and query tabular data without having to rely on any external database engine or package.
Hashes satisfy all four requirements quite nicely.

Had you included this new requirement, I would have gone with one of the CPAN modules, even though that violates your fourth requirement.

Replies are listed 'Best First'.
Re^4: perl database question
by Jenda (Abbot) on Apr 26, 2008 at 21:57 UTC

    Beg your pardon? Hashes support just one type of lookup. Exact match. Please show me how do you store data in hashes so that you can efficiently select all rows in which the value of a certain column is between X and Y? Or those in which a different column contains data that start by ABCDEF? Also keep in mind that in a hash you only have one key ... most likely when working with tabular data you do not always search by the same column, even if you did always ask for an exact match.

    I think he meant "external" to Perl ... some piece of software completely unrelated to Perl that has to be installed. So DBI+DBD::SQLite does fit the bill.

      I think he meant "external" to Perl ...
      A lot of the initial discussion was certainly predicated on assumptions; there was not a lot of detail in the request.

      For example, I assumed we were talking about half a million or less records, each with a primary key and possibly a secondary key. Given my assumptions, it's not too painful to use a Hash of Hashes. For exact queries, the hash is a Big Win. For a partial query, doing a foreach through the hash isn't that painful.

      As soon as the OP mentioned he was talking gigabytes of data, I withdrew my original suggestion.

      So DBI+DBD::SQLite does fit the bill.
      I agree completely.