in reply to Re^6: tying a hash from a big dictionary
in thread tying a hash from a big dictionary
Ouch. They must be some long phrases. Even if you're on a 32-bit OS whereby only 2GB or 3GB of that memory is available to the process.
You're definitely going to need to use external storage.
I used BerkeleyDB with some success for a bit under 32-bit Perl. Though it can take quite a while for the initial building of a large DB, once built the access/retrieval times are about as good as I've ever seen for a disk-based system. You do need to pay some attention to the various configuration parameters to get the best out of it. Look for the BerkeleyDB tuning guide on-line as the module pod is pretty light on tuning.
If you find yourself up against it performance-wise then the object interface is marginally quicker than the tied interface, but much less nice to use. If it is a reference-only DB, sticking (the pre-built) DB file on a cheap SSD, or even a fast thumbdrive, can do wonders for access times.
Just wish I could get it to build for my 64-bit system :(
|
|---|