There's a big difference between DBI and DBM::Deep though. DBI is for talking to a relational database. DBM::Deep is for storing and accessing a perl data-structure on disk. Both have situations where they are clear winners.

The difference between DBM::Deep and Storable is a bit more subtle. Both are for storing and accessing perl data-structures on disk. DBM::Deep gives you random access to that structure without having to pull it all into memory, at the expense of being quite slow for small data sets. For small structures of only 4000-ish elements like what the OP has, the overhead of DBM::Deep appears to be very large. But when you have millions of elements, you'll find that DBM::Deep is faster.

To read, change, and write an element in a Storable file, you need to read the entire file, update that one element, and write the entire file. Reading and writing tens of megabytes is slow. To read, change, and write an element in a DBM::Deep file, the size of the file is irrelevant, you just need to do a handful of small reads and seeks to find the right place, then read, and write just a few bytes. To a good approximation, you will need to read ten bytes for each level of nesting from the root of the data structure to the point which you want to edit, and have one seek per level.

In reply to Re^2: DBM::Deep overhead by DrHyde
in thread DBM::Deep overhead by perlpipe

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.