js1 has asked for the wisdom of the Perl Monks concerning the following question:

I'm using the MLDBM in my script to store 2 hashes:
$staffdept{$staff}
$appstaffwk{$app}{$staff}

[download]

Here's my code:
use MLDBM qw(DB_File Storable);
tie %staffdept, 'MLDBM', $staffdeptfile, O_CREAT|O_RDWR, 0640 or die $
+!;
tie %appstaffwk, 'MLDBM', $appstaffwkdb, O_CREAT|O_RDWR, 0640 or die $
+!;

[download]
Since I started using MLDBM though, it's really slowed down my script. Is this normal or am I doing something wrong?

Thanks for any help.

js1.

Replies are listed 'Best First'.
Re: MLDBM slows down my script
by edan (Curate) on Oct 30, 2003 at 12:01 UTC

    MLDBM is bound to slow down your script somewhat, due to the nature of what it's doing for you (storing your multi-level datastructures on disk). If you read the WARNINGS section of the MLDBM doc, you'll see that it does matter how you access your MLDBM-tied hashes.

    In particular, this means that instead of doing this:

    for my $k (keys %{$h{something}}) {
    print $h{something}{$k}[0]{foo}{bar};  # FETCH _every_ time!
}

    [download]

    .. you're better off doing this, to avoid FETCH-ing the top-level value every time:

    my $root = $h{something};                  # FETCH _once_
for my $k (keys %$root) {
    print $k->[0]{foo}{bar};
}

    [download]

    (Examples taken from MLDBM doc)

    You might want to review your code for these kinds of situations.

    HTH

    --
    3dan

[reply]
[d/l]
[select]
      Thanks for your reply but I'm not fetching anything here. I'm just storing the data for now.

      js1
[reply]
Re: MLDBM slows down my script
by perrin (Chancellor) on Oct 30, 2003 at 16:43 UTC
    Slower than what? In-memory hashes? Of course it's slower -- you are writing all of your data to disk when you use MLDBM.
[reply]
      My script usually takes 1 minute to run, but with mldbm it takes anywhere from 15 minutes to 1 hour (seems quite long even if it is writing to disk). The script creates 3 db files which grow to about 800kb each.

      This is my function to store the HOHs:

      sub store{
      my ($app,$staffno)=@_;

      my $tmp=$appstaffwk{$app};
      $tmp->{$staffno}=1;
      $appstaffwk{$app}=$tmp;

      $tmp=$appstaffmnth{$app};
      $tmp->{$staffno}=1;
      $appstaffmnth{$app}=$tmp;

      $tmp=$appstaffyr{$app};
      $tmp->{$staffno}=1;
      $appstaffyr{$app}=$tmp;
      }

      Any ideas how to make this more efficient?

      Thanks,

      js1.
[reply]
        I don't think there's any way to speed that up easilly. Every time you assign to one of these hashes, it calls an object method, serialized the data with Storable, and writes it to disk. One thing that matters though is what dbm file you are using. If the individual values are pretty small, you can use SDBM_File, which is the fastest. If they are too big for that, use DB_File.
[reply]

Back to Seekers of Perl Wisdom