merlyn's making a good point. This really sounds like a job for a database.

But if you MUST use the filesystem, then yes, you'll definitely need to do something multilevel. Any operations on directories tend to suffer badly when the file count gets high, and "high" in this context is on the order of 10,000's, not millions.

I'd strongly suggest NOT doing "op/us/cows" "optimization". If the full filename is in the leaves, a lot of operations get simpler (since you don't need to remember the path to the file).

And if your filesystem ever got corrupted, you'd never be able to recover - you might be left with an orphaned directory full of files named "cows","goats",etc... with no way of knowing that they belong under op/us. With full filenames, you can survive "mid-tree" corruption without issues, assuming fsck rescues the orphaned data.


Mike

In reply to Re: (OT) should i limit number of files in a directory by RMGir
in thread (OT) should i limit number of files in a directory by leocharre

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.