in reply to BerkeleyDB and large datasets

You should go to Sleepycat's website and read up on the documentation paying special attention to the C API (since that what Paul's BerkeleyDB module wraps). I would suspect you should consider altering some of the default caching strategies or another of the tuning options. Be sure not to miss some goodies like Access method tuning.