Thanks a lot! This forum rocks :)
Well, you're pretty much guaranteed to get your money's worth :-)

I like flat files because I can use all my favorite command line tools to manipulate them, without interacting with a database. And I have a lot of insight and control over what is involved in accessing them. As soon as you start aggregating the files because they may be "grabbed together", then some of the advantages of flat files start getting outweighed by the additional structure needed to aggregate them. Don't rule a real database out, they are good at such things.

Opening a file is not much different from searching a directory... you have to get the contents, and for flat files, that means going out to disk. If you keep related messages in the same file, you may be able to save disk accesses by getting more than one message with one read. But now you've made maintaining the collection of messages more difficult, and searching more complex, because you have different messages in the same file. A database may well be able to do this better.

As with directory structure, try to hide the implementation details. Start with single files, aggregating common files under a single directory. If the performance is good enough, this is likely to be a lot simpler than storing multiple messages in a single file. Above all, keep the user interface simple. They don't want to know about implementation details. And don't dismiss a database out of hand. It may make your life a lot easier, and perform at least as well as a cleverly crafted flat-file implementation

In reply to Re^5: Design flat files database by jpl
in thread Design flat files database by AlfaProject

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.