Re: DBM::Deep overhead

If you want to learn about handling persistent data in Perl, a much better way is to invest some time in reading about the DBI framework. That is more or less the standard way of dealing with all kinds of databases in Perl and for sure will come in handy later. Once you see how easy Perl can handle all kinds of databases, you will discover more and more ways to apply that knowledge.

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Comment on Re: DBM::Deep overhead

Replies are listed 'Best First'.
Re^2: DBM::Deep overhead by DrHyde (Prior) on Apr 21, 2011 at 10:03 UTC
There's a big difference between DBI and DBM::Deep though. DBI is for talking to a relational database. DBM::Deep is for storing and accessing a perl data-structure on disk. Both have situations where they are clear winners. The difference between DBM::Deep and Storable is a bit more subtle. Both are for storing and accessing perl data-structures on disk. DBM::Deep gives you random access to that structure without having to pull it all into memory, at the expense of being quite slow for small data sets. For small structures of only 4000-ish elements like what the OP has, the overhead of DBM::Deep appears to be very large. But when you have millions of elements, you'll find that DBM::Deep is faster. To read, change, and write an element in a Storable file, you need to read the entire file, update that one element, and write the entire file. Reading and writing tens of megabytes is slow. To read, change, and write an element in a DBM::Deep file, the size of the file is irrelevant, you just need to do a handful of small reads and seeks to find the right place, then read, and write just a few bytes. To a good approximation, you will need to read ten bytes for each level of nesting from the root of the data structure to the point which you want to edit, and have one seek per level.	[reply]
Re^2: DBM::Deep overhead by perlpipe (Acolyte) on Apr 21, 2011 at 02:33 UTC
One of the things I love about Perl is that there are always new things to learn and explore as you dig deeper and deeper. Thanks. Will look at the DBI documentation.	[reply]