in reply to Re: Concurrent access with BerkeleyDB
in thread Concurrent access with BerkeleyDB

Thanks for the feedback. That's what I've read but it's not what I'm seeing in my tests which fork two children and try to do concurrent inserts. I can watch the database flying through the inserts and then eventually get hung and never (well at least within 15 minutes ;-) release.

I even went to the trouble of enabling transactions but that did not prevent the lock from occurring. I have had some initial success with limiting the input via sleep 1 (too slow for effective testing) or select (undef,undef,undef,0.25) as described in the perlfaq8.

This "solution" seems more like a hack for an inefficient locking system. Is this a limitation of BerkeleyDB.pm or in the BSDDB libraries? Or perhaps it's my poor understanding of forking which is at fault.

My concern in getting this right is that I have a client with a large database of records which I'd hate to corrupt (I'm weaning them away from Python and corrupting the database would not be a good thing for proving Perl's effectiveness). I've seen several posts by MoveableType users who have had corruptions with BDB. Most of the data is recoverable but the downtime is both annoying and expensive (when it's a commercial business).

Yes, I am considering moving the client to stronger database solution such as PostgreSQL or MySQL. I just wanted to fully exhaust this line of reasoning before recommending the switch. Sleepycat has done some nice work with BerkeleyDB but if it can't do reasonable concurrency then I won't risk using it in a production environment.

I'll post my test script in Meditations in the hope that someone can point out the flaw in my logic or in my implementation of BerkeleyDB.

Wm