There is also the issue of when do you need database-like functions, such as a query language, locking, or management of lot of data with presorted indices. The Berkeley DB site at sleepycat has an interesting What is/is not Berkeley DB? section. The is not page may be useful, because while BDB is a hugely powerful system that can handle terabytes of data, it is not SQL or relational, though you could build a database engine on top of it. This is a pretty interesting page, and it might make you start thinking about extending the life of your system..

Also I would expect the <2GB limitation on machines that only use 32 bit addressing for your plain old flat file, but this may not apply to BDB.. their page says

Because both keys and values can be up to four gigabytes in length, a single record can store images, audio streams, or other large data values. Large values are not treated specially in Berkeley DB. They are simply broken into page-sized chunks, and reassembled on demand when the application needs them.
Now looking at Database Limits it looks like 2GB per database file would be correct for BDB on a 32 bit filesystem machine.

Actually now I started wanting to read it from the beginning and maybe you will too. Hint, first page mentions flat files and structured data!

In reply to Re: When is a flat file DB not enough? by mattr
in thread When is a flat file DB not enough? by TrinityInfinity

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.