The amount of data isn't the problem here. It's how you'd be using it. If you can get it into a "hash" model (one key, one row of values), then a DBM is just fine. However, if you'll ever want to "index" some other value in the tuples, you might as well get into the full-on database instead.
-- Randal L. Schwartz, Perl hacker | [reply] |
merlyn is totally right, but lemme elaborate. Looking at your types of data (large amounts of text and binary) the Berkeley system is going to be way too small for what you're working with. Think of the Berkeley DB as a hash table that's stored to disk; it's great for some things, but comparing it to a relational database is like comparing a car to an F-16.
With MySQL you'll be able to store massive amounts of text data in TEXT columns with virtually no limit - you can do the same for binary data using BLOB. Be warned though - coding for MySQL using the DBI driver is a lot trickier than using the Berkeley DB interfaces.
| [reply] |
I would concur with other replies in that MySQL will probably be the best choice for anything other than simple data requirements.
However, I would differ in my estimation of how much more difficult it will be than working with DBM. Once you get going, it's really not that hard at all.
My recommendation would be to find someone who has worked with Perl and MySQL before (there are loads of them out there -- check a local Perl Mongers group if you don't have a friend or friend of a friend who can help or send me an e-mail if you are really stuck), have them help you set up a database, and run a sample query.
It would take about an hour to step through it all from downloading MySQL to getting the results of your first sample query. And using MySQL will give you more real-world practical experience that you can apply later to more complex problems.
Hope this helps :-)
| [reply] |
I'm not sure that I would call MySQL an F-16. More of a cropduster.
()-()
\"/
` | [reply] |
I was intrigued by this post because you seem to prefer the
use of the BerkeleyDB and are only considering the
MySQL method if you are given compelling enough reason, or
at least that is what I inferred.
We however are not given enough information in my opinion
to give you a compelling reason in either direction. You
express concerns about running the MySQL daemon, but you
really don't get into the nature of the data you are working
with.
The lifespan, stability, and interaction over the lifespan
would effect what is the better way to store the information.
That said, I would recommend the MySQL storage method because
I feel/think it offers less/easier modifications
during the data lifespan for most record oriented material.
| [reply] |