Re^3: (OT) should i limit number of files in a directory

I still think merlyn is right -- a db is the way to go. blobs are not the most elegant/efficient mechanisms, but they are very easy to find based on a key. As long as your blobs stay below about 1MB, mysql or postgres should be fine. Trying to find a single file in a directory hierarchy of millions of entries is going to suffer significantly worse performance.

Comment on Re^3: (OT) should i limit number of files in a directory

Replies are listed 'Best First'.
Re^4: (OT) should i limit number of files in a directory by leocharre (Priest) on Sep 11, 2008 at 17:39 UTC
What about a berkeley db ? That seems like it would be good backend here?	[reply]
Re^5: (OT) should i limit number of files in a directory by Illuminatus (Curate) on Sep 12, 2008 at 14:55 UTC
From everything I have heard, BerkeleyDB would probably be fine as well. I did not mention it because I have not personally used it. I have used both mysql and postgres, so I feel more comfortable recommending them. BerkeleyDB is fast, but has a couple of drawbacks. I don't believe it has network access, so if you want to access the database from a different system, it might not be the best choice.	[reply]