Re: Disk based hash (as opposed to RAM based)

I have looked around more and found http://search.cpan.org/~chorny/MLDBM-2.04/lib/MLDBM.pm

This seemed to do the trick with a bit of modification to my code. However now an error 22 is output, which means that the values I am storing under keys is too large. I suppose it is worth mentioning that I my data is "wide" in the sense that each key will have 1000 or so values that I hoped to store on the array.

     use MLDBM; # this gets SDBM and Data::Dumper
    use Fcntl; # to get them constants
    
    my $dbm;
    my %hash;
unlink ('testmldb.dir');
unlink ('testmldb.pag');

    $dbm = tie %hash, 'MLDBM', 'testmldbm', O_CREAT|O_RDWR, 0640 or di
+e $!;

        while($html =~ m/example(key)regex(value)example/ig ) {
        my $tempArray = $hash{$key};
        push( @{ $tempArray } , $value);
        $hash{$domain} = $tempArray;
        undef $tempArray; #Just for fun-zies
        }
[download]

MLDBM docs do say something about berkleyDB not having this limit, because of how it stores the values. (MLDBM apparently uses joined strings as my array instead of a real array) I will look into the berklyDB approach... I was hoping to avoid using a db and just a file, because I might as well just dump it into a temporary table in my database (which the program also uses) I would just like to take advantage of perls amazing hashes if I could.

Comment on Re: Disk based hash (as opposed to RAM based) Download Code

Replies are listed 'Best First'.
Re^2: Disk based hash (as opposed to RAM based) by davido (Cardinal) on Oct 08, 2012 at 00:56 UTC
I think BrowserUk's mention of DBM::Deep is on track. It should be able to handle whatever level of depth you want to throw at it, and arbitrarily long key lengths. The fact that it's written in pure-Perl is a good feature, as its compatibility with Perl's flexible data types should be higher than is sometimes symptomatic of solutions built in the more rigid environment of lower-level statically typed languages. Dave	[reply]

Replies are listed 'Best First'.

Re^2: Disk based hash (as opposed to RAM based)
by davido (Cardinal) on Oct 08, 2012 at 00:56 UTC

I think BrowserUk's mention of DBM::Deep is on track. It should be able to handle whatever level of depth you want to throw at it, and arbitrarily long key lengths. The fact that it's written in pure-Perl is a good feature, as its compatibility with Perl's flexible data types should be higher than is sometimes symptomatic of solutions built in the more rigid environment of lower-level statically typed languages.

Dave

[reply]