CDB_File memory leak

devnul has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

I seem to be having an issue with CDB_File (0.96) I've reproduced this on OS-X 10.5.2, FreeBSD 7.1, and SuSe 10x, so I think it must be in the module and not system related. (Or, maybe, I'm using it wrong).

At any rate, here is some code:

use strict;

use CDB_File;


my $cdb_file = 'test.cdb';
my $cdb = CDB_File->new($cdb_file, "$cdb_file.tmp") or die "Unable to 
+create CDB file: $!";
while(1)
{
  $cdb->insert('leak', 1);
}
[download]

When this script runs the process's memory footprint grows larger and larger. I thought it might be related to re-inserting the same key over-and-over, so I changed it to something like 'leak'.$i and observed the same behavior.

I reported this as a bug at rt.cpan.org, but thought maybe I'd post it here in case other's had (or have, or will have) a similar issue and someone has a solution.

Thanks, as always!

- dEvNuL

Comment on CDB_File memory leak Select or Download Code

Replies are listed 'Best First'.
Re: CDB_File memory leak by runrig (Abbot) on Apr 29, 2009 at 19:56 UTC
From the docs: A cdb file can contain repeated keys. If the insert method is called more than once with the same key during the creation of a cdb file, that key will be repeated. Update: nevermind...this doesn't explain a memory leak..unless the cdb file was in memory (or is it until the finish() method is called?) :-(	[reply]
Re^2: CDB_File memory leak by devnul (Monk) on Apr 29, 2009 at 20:13 UTC
You might be right.... I never peeked under the hood to see how its doing its stuff, I bet based on what I'm reading in CDB_File::Generator, that it sorts the keys.. Maybe thats why its so fast, after all... - dEvNuL	[reply]
Re: CDB_File memory leak by perrin (Chancellor) on Apr 29, 2009 at 20:36 UTC
Not a direct answer to your question, but BerkeleyDB is both faster and more widely used if you're looking for an alternative.	[reply]
Re^2: CDB_File memory leak by devnul (Monk) on Apr 30, 2009 at 01:23 UTC
Thanks for the suggestion... My experience as well as this PDF would seem to indicate that CDB when working with read-only data (as I am here) is the fastest thing out there. - dEvNuL	[reply]
Re^3: CDB_File memory leak by perrin (Chancellor) on Apr 30, 2009 at 03:27 UTC
Somewhat old, but DBD trounced it in this one. You can also adjust the cache size with DBD, which often makes it faster on systems with a lot of RAM.	[reply]