I think that one of the reasons so few programmers bother to try and index flat file databases is that they view them as a temporary measure until they can implement a "real" DBMS.

Your points are well taken, but I have specific reasons for wanting to use flat text files, and it would be beneficial to make the system as efficient as is practical, given the limitations you describe. Some users may well expand on the basic program, so I wish to create a resilient core.

In this particular case, the database itself would only ever be modified by a single administrator. Modifying existing records would likely be a case of reading in the existing file a record at a time and writing them to a new (modified) file. Once that process is complete, the original file would be renamed to become a backup, and the new file renamed to replace the original file.

This solves the problem of having to fit the entire database in memory, and also creates a backup. File locking would still be a good idea, of course.

Your idea of using MySQL to create flat text files is intriguing (and one I may consider for future projects), but unfortunately the program is intended for users who do not have access to an external RDBMS.

__________
"Every program has at least one bug and can be shortened by at least one instruction -- from which, by induction, one can deduce that every program can be reduced to one instruction which doesn't work." -- (Author Unknown)

In reply to Re: Answer: Where is the bottleneck in flat file databases? by Wysardry
in thread Where is the bottleneck in flat file databases? by Wysardry

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.