Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm doing some work with DBM (dbmopen), and I've run into an issue with portability. I made some DBMs on my "development" computer, and will be moving them to another computer shortly. I tested one of the files out on my primary computer to make sure it'd work okay. Nothing fancy here, just single-dimensional, non-nested hash. When I try to access an element through a known key (in this case, the key is "subject"), I get an error from Perl:

bash-2.05a$ perl
dbmopen(%DBM,"someDBM",undef) or die "$!\n";
print $DBM{subject},"\n";
Use of uninitialized value in subroutine entry at /usr/lib/perl5/i386-
+linux/DB_File.pm line 262.
Use of uninitialized value in subroutine entry at /usr/lib/perl5/i386-
+linux/DB_File.pm line 262.

[download]


The DBM is accessed just fine through my development computer. It's Slackware 8.0; my primary computer (the one that it fails on) is Slack 8.1. Both are running Perl 5.6.1 with nothing added on (except for the LWP modules). Anyone know why it's not opening for me? Do I need to change something? Maybe a different implementation? (I've heard something about IDBM, SDBM, but can't find any documentation that gives me relevant information.)

Thanks!

Replies are listed 'Best First'.
Re: DBM Portability?
by hossman (Prior) on Oct 03, 2002 at 06:46 UTC
    DBM files tend to be very sparse. Depending on how you copy the files from one machine to another, you might be corrupting the file.
[reply]
      In addition to the comments by hossman: although not experienced personally I've heard from a very good source that tar doesn't deal with sparse files properly: use dd and cpio instead.

      rdfield

[reply]
[d/l]
[select]
Re: DBM Portability?
by dree (Monsignor) on Oct 03, 2002 at 09:02 UTC
    With DB_File I had the same problem: a big hash (>150Mb) generated on a Win environment had been read by Perl on Slackware 8.0 but NOT on Freebsd 4.3 ...

    If you are you using DB_File, you can install the same DB_File version on both machines (different version of the same module could have introduced differences with big hashes).

    Another obvious way is to create the dbm again on the second O.S. .
[reply]
      I had that problem once (perl upgraded along with DB_File), and then read in the documentation
      Note: The database file format has changed multiple times in Berkeley DB version 2, 3 and 4. If you cannot recreate your databases, you must dump any existing databases with either the db_dump or the db_dump185 utility that comes with Berkeley DB. Once you have rebuilt DB_File to use Berkeley DB version 2 or greater, your databases can be recreated using db_load. Refer to the Berkeley DB documentation for further details.

      ____________________________________________________
      ** The Third rule of perl club is a statement of fact: pod is sexy.

[reply]
Re: DBM Portability?
by derby (Abbot) on Oct 03, 2002 at 12:41 UTC
    The Cookbook has a nice overview of the different DBM implementations and their features (p90). Here's the table

    FeatureNDBMSDBMGDBMDB
    Linkage comes with Perlyesyesyesyes
    Source bundled with Perlnoyesnono
    Source redistributablenoyesgplyes
    FTPablenoyesyesyes
    Easy to buildN/Ayesyesok
    Often comes with Unixyesnonono
    Builds ok on UnixN/Ayesyesyes
    Builds ok on WindowsN/Ayesyesyes
    Code sizedependssmallbigbig
    Disk Usage dependssmallbigok
    Speed dependsslowokfast
    Block Size Limits 4K1Knonenone
    Byte order independentnononoyes
    User-defined sort order nononoyes
    Partial key lookups nononoyes

    There are other caveats and info so check out chapter 14 of the Cookbook.

    -derby

[reply]
      This is actually totally wrong about the relative speeds. SDBM is the fastest by a mile. There are benchmarks in the readme for MLDBM::Sync.
[reply]
Re: DBM Portability?
by Anonymous Monk on Oct 03, 2002 at 11:30 UTC
    Your problem is that you are using dbmopen. It uses the first of several possible local dbm implementations. Which means that going from one machine to another its file format does not remain the same. It also means that on a single machine if you install a better dbm implementation, the file format it uses can change on you, causing loss of data.

    And no, I have no idea why there aren't BIG RED WARNINGS everywhere about this pitfall.

[reply]
      So if I don't want to go through the hassle of recreating DBMs when I copy from one computer to another, and if possible, not to worry about checking versions on all of my computers to make sure they're the same, and I don't want to hassle with using dd or cpio, what options to I have?
[reply]
        1. Make sure you are using the same format on each machine. You could use tie %hash, DB_File, 'file'; which specifies which DB type you want to use explicitly (unless you use AnyDBM_File :). Also, dbmopen has been deprecated since Perl 5.
        2. Try compressing the data file and then transferring, or, if there's more than one, compress all of them first, and then tar them together.

        --
        hiseldl
        What time is it? It's Camel Time!

[reply]
[d/l]

Back to Seekers of Perl Wisdom