If you need to do this in the filesystem, which merlyn makes good points against, then you definitely don't want to steal characters from the filenames as RMGir says.

You probably also want to use "/op/opus/opusco/opuscows" rather than just "/op/us/co/opuscows" too, for the same reasons. It's possible to rebuild the whole directory system based on just the file names as long as the file names are intact, but the directory names being salvageable as well will help when something goes wrong.

Is there any data in these files? Are these hash-named files the files you're checking for existence, or are you using them to track the existence of other files? 16**32 is much larger than 3 million, so I'm guessing these are hashes of other files.

Is this a tracking system to see if files have been inserted into a document management system? If so, you'll have issues if the documents are editable because the MD5 sum will change. You'd have to delete the hash for the old version before the edit starts and recreate it after the edit every time. It might be easier to store both the document and the hash for it in a database if you're doing something like that.

In reply to Re: (OT) should i limit number of files in a directory by mr_mischief
in thread (OT) should i limit number of files in a directory by leocharre

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.