I'd imagine Storable is faster but here's something fun to look at.

You might like to check Lincoln Stein's page on Boulder.

"Boulder IO is a simple TAG=VALUE data format designed for sharing data between programs connected via a pipe. It is also simple enough to use as a common data exchange format between databases, Web pages, and other data representations."

I have not benchmarked the serialization, but his Wormbase handles 2 gigabytes of genetic data and seems pretty quick.

With Boulder IO you can make nested data structures, and you can just pull the tags you want out of a boulder stream letting other tags pass through to other programs in the pipeline. A boulder is made of a stone which can hold smaller stones; stones are stored in a berkeley dbm. Don't know how tough it would be to roll support for another db. From the docs it would seem that with the tags() method you can glean some information from the ascii tag of a deeply nested stone without unpacking its data.

Thought this might be a conceivable way to share data.. but do note that a Fcntl lock keeps users from reading while a write is going on.

Anybody else with experience using boulder io, would love to hear it.