Re: A more memory efficient storage structure?

Hello again;
I grabbed Devel::Size, and according to it, each of the hashes is 6.1M in size.
'top' shows the memory usage at 18M, and I don't think the interface is using 6M of space... So I'm not entirely convinced thats quite correct. But it stills gives an idea of the inflation - 727k to 6.1M.
I've never been able to make BerkeleyDB install correctly (Out of two tries, I believe), so I'm not terribly interested in it out of previous predjudices, but I could certainly entertain using an ondisk DB... I can't imagine it'd be terribly quick for the lookups, but I'll never know until I try.
(My workstation runs Linux, if that tells anyone anything)

JP,
-- Alexander Widdlemouse undid his bellybutton and his bum dropped off --

Comment on Re: A more memory efficient storage structure?

Replies are listed 'Best First'.
Re: Re: A more memory efficient storage structure? by Elian (Parson) on Dec 31, 2002 at 19:22 UTC
Do be aware that Devel::Size does count shared data in part of its size calculation when figuring out how big what you're checking is, but it only counts it once per call, so if the two hashes share some data, and they might, pass them in together for a correct aggregate total. Something like: `print total_size([\%hash1, \%hash2]);` [download] though that'll add in a few dozen bytes for the two refs and the anon array. Also, don't rule out the possibility that there's a lot of scrap memory hanging around that Perl's not cleaned up after quite yet, or that you've not noticed.	[reply] [d/l]
Re: Re: A more memory efficient storage structure? by shotgunefx (Parson) on Dec 31, 2002 at 19:48 UTC
You only need BerkeleyDB if you want the newer features (transactions,record locking,etc). Perl ships with BerkeleyDB, look in the directory ext/DB_File for the version. Getting newer versions to install was a pain in the a**, the problem it turned out for me was which db.h file was being included in the compilation, so if you want the newer features, that's a good place to start looking for problems. -Lee "To be civilized is to deny one's nature."	[reply]
Re: Re: A more memory efficient storage structure? by perrin (Chancellor) on Dec 31, 2002 at 22:18 UTC
SDBM_File is actually much faster than DB_File or BerkeleyDB when dealing with a single process, and it comes standard with Perl on most systems. However, if speed is the most important thing, you should just stick with what you have. The memory inflation is not unusual.	[reply]