Short answer: too many variables to tell. You should be ok, though. I've got a CMS that sometimes has to read two megabytes of xml and manages it without becoming unusable. It's not the right way for me to be doing it, but it just grew, and it still works ok. I suspect that it will be issues of data integrity or structure that make you shift, not capacity.

Anyway, if you use DBI and something like DBD::CSV to manage access to your flatfiles, then you'll have a ready-made layer of abstraction between the scripts and the data. in which case it kind of doesn't matter about the ceiling: if you seem to be reaching it, you can switch to something with more headroom just by changing a line or two in the top of the script (and getting the IT people to provide a bit of access to the databases. Not in that order.)

There are useful discussions of DBI and DBM around, if you want to look into that further.

In reply to Re: When is a flat file DB not enough? by thpfft
in thread When is a flat file DB not enough? by TrinityInfinity

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.