Berkley DB version problems; work stoppage!

kleucht has asked for the wisdom of the Perl Monks concerning the following question:

I was forced to move to a new server recently and now I cannot get my web-based application to work anymore. (There are people getting real upset with me now.) The old application running on the new server is not able to open the old Berkley DB file (on the new server) and I also am not having any luck db_dump'ing it either. I'm not real familiar with Berkley DB's, I just use them sometimes. Please help me if you can and I will be eternally grateful!!! (I should also mention that I no longer have access to the old server, which may be a problem but I'm hoping not.)

Here are some technical details from the new server that I hope will tell you exactly what the problem is:

[kurt@amber]$ ll
total 436
-rw-------    1 kurt     kurt        12288 Nov 22 22:09 new.db
-rw-r--r--    1 kurt     kurt       212992 Nov 22 21:35 old.db
[kurt@amber]$ file old.db
old.db: Berkeley DB (Hash, version 5, native byte-order)
[kurt@amber]$ file new.db
new.db: Berkeley DB (Hash, version 7, native byte-order)
[kurt@amber]$ db_dump old.db
db_dump: old.db: hash version 5 requires a version upgrade
db_dump:open:old.db:DB_OLDVERSION:Database requires a version upgrade
[kurt@amber]$ db_dump185 old.db
db_dump185: old.db: Invalid argument
[kurt@amber]$
[download]

NODE UPDATE Thanks to diotalevi for pointing out that the db_upgrade utility was staring me right in the face! Thanks for the help! I'm up and running again!!!

Comment on Berkley DB version problems; work stoppage! Download Code

Replies are listed 'Best First'.
Re: Berkley DB version problems; work stoppage! by diotalevi (Canon) on Nov 23, 2003 at 04:02 UTC
http://www.sleepycat.com/docs/api_c/db_upgrade.html covers the C API call DB->upgrade. I'd guess you'll need to write a short C program to call this for you or use something like Inline::C to do it in mostly-perl.	[reply]
Re: Berkley DB version problems; work stoppage! by tilly (Archbishop) on Nov 23, 2003 at 17:17 UTC
For future reference, in addition to your database, keep a plaintext backup that you refresh periodically. (Just write a script to open your dbm, walk the keys and dump it in a plaintext format.) That way if you are dealing with a problem like this, you have the option of writing a script to take the plaintext backup and recreate the database. Actually you should do this before you run into trouble, and try the dump/undump utility out. (That way you can find that, for instance, your plaintext format assumes no returns in the data, but there are returns...) And a note for the future. What happened to you illustrates why in a switchover people should ALWAYS do testing (which should have caught that your application had problems) and also keep the old server around (so that people can figure out how to rescue stuff that broke). Also a random tip. If you are using this in a CGI environment, be warned that the widely used locking technique of calling flock on the dbm does not work. (This used to be the documented approach, and appeared, for instance, in the old Cookbook.) Use either the native locking facilities in BerkeleyDB or use DB_File::Lock instead. (If you are not using locking, of course, then you should. The consequences of not doing so include file corruption and loss of data.)	[reply]
Re: Re: Berkley DB version problems; work stoppage! by perrin (Chancellor) on Nov 23, 2003 at 21:55 UTC
MLDBM::Sync also works for locking.	[reply]
Re: Re: Re: Berkley DB version problems; work stoppage! by tilly (Archbishop) on Nov 25, 2003 at 01:46 UTC
Lots of locking solutions work. DB_File also suggests Tie::DB_Lock and Tie::DB_LockFile. I limited my recommendations to avoid presenting so many options that the questioner would get confused.	[reply]
Re: Berkley DB version problems; work stoppage! by kleucht (Beadle) on Nov 23, 2003 at 04:04 UTC
From the Original Submitter: Additional details that I left out of the original post: The old.db file was created on the old server within a perl script with the following call: `dbmopen(%DATA, $database, 0600)` [download] The new.db file was created on the new server by the same perl script. I don't understand the Version 5 and Version 7 results since Berkley DB files are only version 1, 2, or 3, right? I hope this makes sense to someone, because it sure doesn't make any sense to me! Thanks for any help you can give, Kurt	[reply] [d/l]
Re: Berkley DB version problems; work stoppage! by kleucht (Beadle) on Nov 23, 2003 at 04:11 UTC
From the Original Submitter: Is there an executable conversion utility or script out there somewhere that I can just execute by hand? I don't know anything about Inline::C	[reply]