in reply to Giant Tie'd data structures

A tied hash is just one of many dozens of ways of doing this. Maybe you settled on the technology prematurely before considering the consequences. What are your real requirements? What will you be doing with the data? How are you accessing it? Maybe it's time for a real database (like DBD::SQLite).

-- Randal L. Schwartz, Perl hacker
Be sure to read my standard disclaimer if this is a reply.

Replies are listed 'Best First'.
Re^2: Giant Tie'd data structures
by mast (Novice) on Oct 26, 2005 at 00:14 UTC
    I need something that can store, and retrieve, items of data with specific keys (which themselves store important data.)

    I've just tried SQLite and it appears to be orders of magnitude slower, but I'm sure I'm just doing something wrong.

    I know bdb can be used to store large GB of data--so which one should I be using? A Tie'd BTree maybe?

    I'd love to educate myself about what the storage limitations are of the various tie methods. Doh! Help!
[reply]
      SQLite is a lot slower than Berkeley DB. A couple of tips for you:

      The error message says to increase page size. You are setting "bsize" which means block size. The page size parameter is called "psize."

      Also, you can tell it to use a BTree instead of a hash, and this is usually faster as well.

[reply]

In Section Seekers of Perl Wisdom