If you are using spinning disks (not static ram) to store your files and directories, then a useful rule of thumb is that a 7200 rpm disk spins 120 times a second, so each byte rolls by every .0083 seconds. On average, you can do no better than 4 milliseconds to fetch the byte(s) you are after in a random access. (You can get a lot of associated bytes with the same read, so data density helps in transfer time, but not at all with disk latency). With processor cycle times in the neighborhood of a nanosecond, you can execute a lot of instructions in 4 milliseconds. You could search a few thousand bytes of directory data, even using linear search, in far less time than it would take you to access that data. So don't make your directory hierarchy too deep. Each subdirectory is going to cost you at least 4 milliseconds to read. A few levels may get cached, but that's equally true of shallow hierarchies. Keep disk accesses in mind when you design your system.

In reply to Re: Design flat files database by jpl
in thread Design flat files database by AlfaProject

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.