in reply to Re^3: Perl solution for storage of large number of small files
in thread Perl solution for storage of large number of small files

SQLite is a "database of last resort". It's good for prototyping, and for bundling with applications cos it requires no setup, but most of the time I'd prefer to use PostgreSQL, MySQL or a "grown-up" database like Oracle.

Implementing your own primitive bucketing algorithm by using several DBM files sounds ... perverse. If going down the DBM route, I'd use just one file and let perl handle that. I'd also use GDBM_File in preference to DB_File as it has fewer restrictions on your data. You might want to consider DBM::Deep if you want to fake up a filesystem in a file, or go crazy and create a *real* filesystem in a file, optimised for your circumstances, and mount that.

Replies are listed 'Best First'.
Re^5: SQLite performance
by jbert (Priest) on May 02, 2007 at 10:52 UTC
    I disagree. SQLite is a 'database of first resort'. It doesn't scale to the same heights as server-based approaches, but those heights are quite a way off for the majority of systems and it's performance is good when you stay within it's scaling limits. The docs are also up-front about when you *don't* want to use it.

    Using sqlite in a solution simplifies the solution compared to server-based RDBMSs (fewer moving parts and fewer deployment/management issues) and so reduces the costs and time spent dealing with it.

    So unless you:

    1. already have existing server-based database infrastructure
    2. know in advance your expected load is approaching the limits of sqlite
    3. or particularly need a feature one of the server-based RDBMSs provides
    I'd say that choosing MySQL or postgres over sqlite is premature optimisation.
[reply]

In Section Seekers of Perl Wisdom