Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Re: Need directory scheme to store 400,000 images

by liz (Monsignor)
on Apr 12, 2004 at 16:31 UTC ( #344428=note: print w/replies, xml ) Need Help??

in reply to Need directory scheme to store 400,000 images

If you want to do it really simple, use a ReiserFS partition. From the front page blurb:

Do you want a million files in a directory, and want to create them fast? No problem.

I've used ReiserFS in one situation in the past where I just copied a tree structure of about 70Gb onto a ReiserFS file system: in the end, it only used about 55 Gb. Never had any trouble at all. Highly recommended!


  • Comment on Re: Need directory scheme to store 400,000 images

Replies are listed 'Best First'.
Re: Re: Need directory scheme to store 400,000 images
by PhiRatE (Monk) on Apr 13, 2004 at 01:34 UTC
    I agree. Reiser does an excellent job of handling the many-files-in-one-dir problem, avoiding the need for artificial hashing schemes.

    Regarding reliability, I used reiser3 now exclusively on my 2.6.2 laptop and have done every murderous thing possible to it including hard poweroffs in busy-writes and numerous attempts to get software-suspend working (got it in the end but did many horrible things to my system in the process, reiser took it all without blinking).

    I understand that there were problems in the past, and I would highly recommend that solid testing be done prior to deployment on a production platform, but I really do think the solution superior to the directory hashing workaround if the option is available.

Re: Re: Need directory scheme to store 400,000 images
by markjugg (Curate) on Apr 12, 2004 at 19:45 UTC
    Interesting to know. I've personally had bad experiences with ReiserFS, at least as provided by the Mandrake Linux 9.2 installer. On my own machine I got crippling file system errors after just a few days with it. I quit using it at that point. Another friend was using ReiserFS and found it hung at boot time after a few days, needing a manual fsck. I've had less problems with 'ext3'.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://344428]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (2)
As of 2023-09-29 17:54 GMT
Find Nodes?
    Voting Booth?

    No recent polls found