in reply to Re: Re: Table Generation vs. Flat File vs. DBM
in thread Table Generation vs. Flat File vs. DBM

YMMV, depending on which flavor of DBM you pick and how your hardware gets along with it. As a general rule, writing to any sort of DBM file tends to be somewhat more expensive than writing a flat file, in terms of overall space consumed, amount of actual disk i/o performed, and total cpu time required.

But when reading data back after you've stored it, a DBM file is vastly better, especially when fetching values in a quasi-random fashion from a very large set -- or at least, whenever the fetching order is very different from the storage order. In such cases, doing repeated sequential searches over a flat file will kill you, whereas the DBM file is really just a big hash array on disk, optimized to deliver any chosen piece of data in a consistently short amount of time.

So the question really is "what sort of access do you really need when reading the data back?" If you can easily write a flat file such that you just need to read it back once from beginning to end, then a flat file will be the better choice.

  • Comment on Re: Re: Re: Table Generation vs. Flat File vs. DBM