BerkeleyDB locks

tbusch has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, I'm using the CPAN package BerkeleyDB (not DB_File) to share hashes between processes. The Berkeley environment I use for this purpose is initialized with the flags DB_CREATE DB_INIT_CDB, and DB_INIT_MPOOL. It all works fine except when one of the processes crashes during a read or write. In that case the Berkeley Hash gets locked forever. Is there any way to avoid locking altogether when using concurrent access to a BerkeleyDB, so that a crash of a process wouldn't affect the BerkeleyDB ? If not is there a way to let locks expire. And if this doesn't work what are my alternatives. Many thanks in advance. Thomas

Comment on BerkeleyDB locks

Replies are listed 'Best First'.
Re: BerkeleyDB locks by dirving (Friar) on Jul 31, 2006 at 17:52 UTC
I'm not terribly familiar with the way BerkeleyDB's locking works (and it seems like there should be a way to take care of this locking problem from within BerkeleyDB), but the big question for me was what you mean by the process "crashing." If it is just you (or some module you use) calling "die", then you can use an END {} block (see perlmod) to do the cleanup necessary to release the lock on the DB. If it's the actual perl interpreter crashing you have much bigger problems, and you should solve that problem first before worrying about the file locking. Another option might be using a module like LockFile::Simple to lock the hash independent of BerkeleyDB semantics.	[reply]
Re: BerkeleyDB locks by perrin (Chancellor) on Aug 01, 2006 at 00:02 UTC
It is possible to do your own locking, with MLDBM::Sync. However, performance will be much worse than letting BerkeleyDB do it. Have you tried to recover it using the db_recover tool they supply? That normally fixes any leftover lock issues.	[reply]
Re^2: BerkeleyDB locks by mjemmeson (Monk) on Aug 01, 2006 at 09:36 UTC
it should clear the locks, but all processes will have to stop accessing the db while the recover tool is run, which may not be possible...	[reply]
Re^3: BerkeleyDB locks by perrin (Chancellor) on Aug 01, 2006 at 14:40 UTC
When you have a crash and something gets corrupted, you have to stop and recover from it. I don't think there's any way to avoid that.	[reply]
Re: BerkeleyDB locks by Khen1950fx (Canon) on Jul 31, 2006 at 18:17 UTC
Good question! In general, Hash access method page splits can happen at any time, and, when you're dealing with concurrency, there is no way to prevent locking. Probably your best bet is to check the Berkeley DB Reference Guide: Berkeley DB Reference Guide Also, I'd recommend using the Berkeley DB Concurrent Data Store Product. It might solve your problem.	[reply]