With significant caveats, yes, you are wrong.

The key to handling large data sets is to have efficient data structures and algorithms. A dbm does this. Given that there aren't significantly better ones available, a relational database cannot improve much. Oh, sometimes it might be possible to get a slight win from a known fixed structure. Mostly if you do, you lose it several times over from having the relational database. (Particularly if, as is usually the case, you have the database code in another process that your process needs to communicate with.)

However change the picture by saying that you don't have a key/value relationship, but rather have a tabular structure where you want to be able to do quick lookups on either of two different fields. Stuff that into any decent relational database, add an index on those two fields. Done. What do you have to do to get that with dbms? Well, you could store your data in a long linear file, and then store offsets in a key/value relationship in a couple of dbms (one for each index). This is a lot of custom code to duplicate what the relational database is doing already. This is a lot of work to duplicate what the database did already. And should the spec change just slightly (say you need a third field), you have a lot of recoding (and debugging) to do.

Sounds like if you need even the simplest of basic structures, relational databases give a nice development win over a dbm. And should you need 2 tables and a join, well your average programmer will probably program it by searching each table once for each element in the other. (Either explicitly or implicitly.) Avoiding that mistake by default makes your algorithms scale much better. Need I say more about how quickly the relational database pays for itself?

But if your problem is dead simple, then the relational database is a loss.

In reply to Re (tilly) 4: millions of records in a Hash by tilly
in thread millions of records in a Hash by johnkj

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.