STM for Perl

I have just pushed DBM::Deep 0.99_03 to CPAN. This is the final candidate release before I release 1.00. Major changes are improvements in how transactions were done and freespace management. This should be the final step to software transactional memory for Perl.

0.99_03 Jan 22 21:45:00 2007 EDT
    - THIS VERSION IS INCOMPATIBLE WITH FILES FROM ALL OTHER PRIOR VER
+SIONS.
      - The fileformat changed completely. I will be writing a convert
+er, but
        it's not there right now. Do NOT expect that this module will
        correctly detect older versions and handle them sanely. Sanity
+ will be
        there for 1.00, but we're not there yet, are we?
    - Converted to use FileHandle::Fmode to handle filehandle status c
+hecks
    - Fixed bug with deleting already-deleted items on Win32 (reported
+ by Nigel Sandever)
    - The guts of how transactions work has been rewritten to better h
+andle
      some edgecases. This required a complete rewrite of the engine.
    - Freespace management is now in place. It's not perfect, but it's
+ there.
    - The rewrite of the engine required a rewrite of how first_key/ne
+xt_key
      was implemented. This should result in significant speed improve
+ments.
    - Self-reference has been removed. This means you cannot do:
          $db->{foo} = { x => 'y' };
          $db->{bar} = $db->{foo};
      I hope to be able to return this functionality by 1.00, but I ca
+nnot
      promise anything. To do this properly, it requires refcounting i
+n order
      to correctly handle deletions and transactions. Once you move aw
+ay from
      a simple tree, everything becomes really hard.
[download]

My criteria for good software:

Does it work?
Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Comment on STM for Perl Download Code

Replies are listed 'Best First'.
Re: STM for Perl by gaal (Parson) on Jan 24, 2007 at 06:12 UTC
STM is usually used to denote a technique for lockless atomic concurrent access to data. It relies on separating IO from transaction code for correctness and as such is quite hard to do in Perl 5 (although Pugs has a prototype of this working). DBM::Deep might help with an in-memory database (though I thought the point was that it gave a Perl interface to disk backed storage), but how is this STM?	[reply]
Re^2: STM for Perl by dragonchild (Archbishop) on Jan 24, 2007 at 20:40 UTC
DBM::Deep's initial mission was to provide a Perl interface to disk-backed storage. It did this so well that programs don't even know they're working with disk-backed data structures. Combine this with transactions and you have the ability to treat a data structure transactionally. In other words, "lockless atomic concurrent access to data." That this lockless concurrent access occurs on disk is undesirable, but transparent to the program. Adding the ability to do this in-memory vs. on disk is an optimization, not a specification. My criteria for good software: Does it work? Can someone else come in, make a change, and be reasonably certain no bugs were introduced?	[reply]
Re^3: STM for Perl by gaal (Parson) on Jan 25, 2007 at 07:25 UTC
I'm not dissing DBM::Deep. Might be a useful module indeed. But it isn't STM. STM is a particular technique to achieve safe access to data in a multithreaded program. The concurrency it deals with isn't that of different processes accessing the data on disk at the same time, and the locks it saves the programmer from thinking about aren't of the filesystem type.	[reply]
Re^4: STM for Perl by dragonchild (Archbishop) on Jan 25, 2007 at 14:15 UTC
Re^5: STM for Perl by gaal (Parson) on Jan 25, 2007 at 14:40 UTC
Re: STM for Perl by diotalevi (Canon) on Jan 24, 2007 at 06:07 UTC
That's neat. Where's the source repository? I'd like to submit patches (currently just doc and pod syntax stuff). ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊	[reply]
Re^2: STM for Perl by dragonchild (Archbishop) on Jan 24, 2007 at 20:29 UTC
The repos is http://svn.perl.org/modules/DBM-Deep/ - I'm currently working in the stonehenge_cleanup branch. My criteria for good software: Does it work? Can someone else come in, make a change, and be reasonably certain no bugs were introduced?	[reply]
Re: STM for Perl by eric256 (Parson) on Jan 25, 2007 at 17:33 UTC
I've always wanted a Scanning Tunneling Microscope for my perl! Sorry couldn't resist. ___________ Eric Hodges	[reply]
Re^2: STM for Perl by stvn (Monsignor) on Jan 25, 2007 at 20:48 UTC
Don't be silly, it is a module for use while riding the Montreal Public Transit System. Err wait,.. no I think maybe it has something to do with Short Term Memory, but I forgot. Hmm... perhaps Shoot The Messenger?? Damn google is no help at all. -stvn	[reply]