in reply to A memory efficient hash, trading off speed - does it already exist?

It sounds like you might want to use a tied hash, storing most of the data only on disk, most of the time. Another reply mentioned DB_File/DBM_File, and that is basically it. One problem is that tied hashes need to be, basically, flat datastructures. In English, this means that "you can't just throw a HoH at it, and expect it to just work" (sic). You need to turn the subhashes into scalars, and vice versa. Thus, look at MLDBM/Tie::MLDBM, which will take care of this nitty gritty for you.

In case you're really after a way to keep the data in memory (what do you mean, so you don't need persistent data? ;-), the same concept might save you some memory: keep a plain flat hash as your main data storage, of which the values are frozen records, which you must first thaw (to use the Storable terminology) into a plain hash in order to access the deeper underlying data. That's right, you can use Storable for that.

Somehow, I've got the gut feeling that this would imply reinventing MLDBM.

  • Comment on Re: A memory efficient hash, trading off speed - does it already exist?