(updated for clarity)

I've been working on a Closed Process Group/Virtual Synchrony algorithm to provide a shared state/initial state transfer implementation in Perl. It is not yet a fully fledged module, but prior to that any input anyone may have on the best way to develop it would be welcome.

As an example of what it can do I've made a networked multi-master tied hash example. Basically, run a "server" instance on each node in a cluster and (UNIX domain socket) clients can see all the data which other nodes have entered.

Hashes are kept strictly in sync with each other and the data is resilient until all nodes in the cluster have failed (quorum can be added of course).

For example nodes 1 and 2 could concurrently access a shared hash:


node1: printf "> %s", $a{'hello'}
node1: >

node2: $a{'hello'} = "world"
node2: printf "> %s", $a{'hello'}
node2: > world

node1: printf "> %s", $a{'hello'}
node1: > world

Also there's a more dynamic example which emulates Hypervisor nodes managing VM resources - automatically reallocting VMs when an HV goes out of service or comes back in.

Any application which requires nodes in a cluster to have a timely view of the whole cluster state could benefit from this approach.

POD docs are available in the demo scripts over at https://github.com/davidcoles/bogart

If anyone is working on anything similar or might find this work useful then please feel free to get in touch!

  • Comment on RFC: Distributed/replicated TIEHASH and shared state algorithm with Corosync

Replies are listed 'Best First'.
Re: RFC: Distributed/replicated TIEHASH and shared state algorithm with Corosync
by zentara (Cardinal) on Dec 29, 2013 at 11:41 UTC
    Maybe they could incorporate this concept into Git itself? :-) Sort of a realtime updated Git repository? But wait, I thought that one of the advantages of an offline Git repository is that you can work if the line goes down. What sort of re-synchronization process does this have in case a few of the realtime sockets fail?

    I don't work on these things except as mathematical abstractions in my mind. :-)

    After seeing how jquery and ajax work, I beginning to believe that eventually we all will be running these supposedly non-blocking event-loop programs, in all our software, javascript will have won in the end. Every square inch of our screens will be controlled by 1 event-loop or another, dealing with their own sockets. It's called Web 2.0 I believe. :-)

    But if I was working on it, I would ponder how to resynchronize after a communications failure. Like would the histories of all changes, which occurred during the down time, be replayed for the benefit of the central repository tree.

    I was recently watching a youtube of Linus Torvalds, and he said essentially that realtime central systems are not good. Everyone should run independently and update each other on a regular basis.

    Another thing Torvalds talked about, was the way that in Git, the node names are the actual md5sums of the diff data in the node. This assures perfect replication... what comes out is what went in, or error flag.

    But these are just the ramblings of someone having a good christmas. :-)

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
[reply]

      I hope I didn't overload the word "hash" too much - this is as in the Perl sense (%hash, $hash{key}=value) rather than MD5/SHA1 type hashes. Tie your hash variable with this and becomes replicated over the network.

      At the moment it is an in-memory object so more suitable for tracking state of a system rather than a permanent repository, but no reason why it could not use a disk-based backend and do initial state transfers with a separate channel (current implementation limits the full database to be ~1MB in size).

      Resync after failure currently involves a full state transfer, but it could be adapted to checkpoint periodically and transfer diffs (quorum to avoid splitting the cluster would be advisable too). Also it uses multicast so it's more of a local thing rather than general internet use.

[reply]

Back to Meditations