Okay, I would like a little help here. I'm sorting through some data and want to store it in a hash of hashes ($hash{key}{subkey}=$value). No biggie. The problem is that the data exceeds 800MB in size when fully loaded.

So I considered using tie and related functions to store the hash on disk. Of course, you can't have two dimensional hashes via tie. But since my first key is sparse (around 20 entries), I thought I could create a tied hash just for that key, so that the subkey would be on disk.

My attempt looked something like this:
sub checkdbm {
  my $hashref=shift;
  my $key=shift;
  
  if (-f "$key-dbm.db") {
    return;
  }

  tie (%{$hashref->{$key}},'NDBM_File',"$key-dbm",O_RDWR|O_CREAT,0640)
+;

}

[download]
with the invocation of:
  checkdbm \%terms,$db;

  for my $t (@terms) {
    $terms{$db}{$t}++;
  }

[download]
But alas, I can't get it to work. The code executes, the key-dbm.db files are created, but no data is ever stored in it; the program grows in memory until even switching virtual consoles takes minutes to perform.

Can anyone see what my problem is? (Ahem, my *programming* problem). I suspect I don't understand precisely how tie works and that is biting me in my butt.

TIA!

In reply to Tie, DBM's, HoH and Sparse Keys by mr.nick

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.