in reply to Re: MLDBM tie fails
in thread MLDBM tie fails

Thanks for your response. I included the use statements at the top of my post. Sorry, but I don't understand how these are incorrect- can you specify? They look quite similar to yours. Here is the relevant code in my program: 
#!/usr/bin/perl -w
use strict;
use Fcntl qw(O_RDWR O_CREAT); 
use MLDBM qw(DB_File Storable);
....
tie(%positions_hash, 'MLDBM', 'positions.db', O_CREAT|O_RDWR, 0640) or
+ die "Could not create db file\n"; #write positions hash to db file
untie %positions_hash;

[download]

The hash is a large, complex multi-level structure created in several subroutines which I didn't post. I suspect the problem has something to do with its size (> 1G). Your code, which is basically identical to mine except that the tied hash is a single-level small structure, also works fine on my system.

Replies are listed 'Best First'.
Re^3: MLDBM tie fails
by tilly (Archbishop) on Dec 17, 2008 at 01:24 UTC
[reply]
Re^3: MLDBM tie fails
by oko1 (Deacon) on Dec 17, 2008 at 02:58 UTC

    I don't recall seeing them in your original post; did I just miss them, or did you do a "stealth" update? In any case, given that additional datum, my response would be the same as tilly's: MLDBM is reputed to suffer "mysterious failures" when stressed (I tend to avoid it, myself.) Again, echoing tilly, DBM::Deep is a good recommendation.


    --
    "Language shapes the way we think, and determines what we can think about."
    -- B. L. Whorf
[reply]
      Thanks, I'll try DBM::Deep. BTW there were no stealth updates on the first post :-)
      UPDATE: Tried DBM::Deep. I'm getting this error:
      Use of uninitialized value in -e at /usr/local/share/perl/5.8.8/DBM/Deep/File.pm line 55
      Here is the code snippet: 
      #!/usr/bin/perl -w
use strict;
use Bio::AlignIO;
use Bio::SeqIO;
use Fcntl qw(O_RDWR O_CREAT);
use DBM::Deep;
require Variation::RecombConfig;
use Variation::Recomb;
use POSIX qw(ceil floor);
use DirHandle;
use File::Temp;
use Data::Dumper;
# do lots of stuff to create a big complex hash, %positions_hash

my $db = DBM::Deep->new( -filename =>"positions.db", pack_size => 'lar
+ge');
$db->import(\%positions_hash);
$db->optimize();

      [download]
      From the DBM::Deep code it seems that it is not properly creating the new file positions.db. Is there a way around this?
[reply]
[d/l]
[select]
        I would suggest tying the big complex hash to DBM::Deep before creating the big complex data structure. That is because processing the whole thing at once takes up too much memory. Doing it incrementally from the start will behave much better.

        If that fails then try to come up with a minimal test case and file a bug report at http://groups.google.com/group/DBM-Deep?pli=1.

[reply]

In Section Seekers of Perl Wisdom