pashanoid has asked for the wisdom of the Perl Monks concerning the following question:

Dear Bretheren, could you recommend a perl "native" database format/solution that I should implement. My program generates a pretty big log file with 1-2 tables. My boss now wants me to draw charts so I need to access the data I generate. My program is for both Linux and Win32 platforms so using mysql or something that requires "administrative" skills from the user is out of the question. What should I use? Berkley DB file? cvs? dumping a hash to a file? something else? Please note that the log files grow and I need to rotate them at some time... At first my client wanted these data to be saved in an excel file, and I've done that. But now he wants charts and extra things so I have to ask your opinion. Many thanks! God bless!

Replies are listed 'Best First'.
Re: perl native db
by Marshall (Canon) on Aug 16, 2011 at 06:32 UTC
    If you want to use a SQL database without requiring administrative hassles, that means SQLite. SQLite is the DB of choice for most smart phone applications. I use the SQLite Manager plug-in for Firefox to view existing databases on my Windows PC.

    SQLite is a simple: one file <=> one database. There are no passwords or users. If my process can access the DB file, then I can use the Perl DBI to connect to it, read it and modify it.

[reply]
Re: perl native db
by Anonymous Monk on Aug 16, 2011 at 06:23 UTC
[reply]

      Thank you! I'm trying to create a system-wide config area where I'll place a simple config file and the database file, however, I'm getting all kinds of errors (module not installed and etc) 

      #!/usr/bin/perl

use strict;
use File::UserConfig;
#use File::ShareDir ':ALL';
use File::Read;

my $username =  getlogin();
print "user: $username\n";
#my $configdir = File::UserConfig->configdir;
my $configdir = File::UserConfig->new(
        dist     => 'Map-Energy',
        dirname  => '.map_energy',
        module   => 'Map::Energy',
)->configdir;


#my $dir = dist_dir('map_energy');
#my $config_file = dist_file(  'map_energy',  'config/data.txt');
#my $db_file = dist_file(  'map_energy',  'db/mapdata.db');
#my $mapsettings = read_file({ as_root => 1, skip_comments => 1, skip_
+blanks => 1 }, $config_file);
#print "map settings: $mapsettings\n";

exit;

      [download]
[reply]
[d/l]
        There are "all kinds of solutions" for "getting all kinds of errors" ...

        Seriously, why don't you start installing the missing modules? Fire up old trusty cpan and see your problems disappear.

        CountZero

        A program should be light and agile, its subroutines connected like a string of pearls. The spirit and inte>nt of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

[reply]
[d/l]
Re: perl native db
by locked_user sundialsvc4 (Abbot) on Aug 16, 2011 at 12:20 UTC

    I, too, would recommend SQLite ... and let me make one very important point:   you must use transactions.   It will make quite a dramatic difference in performance, and here’s why.   If a transaction is in progress, SQLite will do “lazy writes.” If not, it will verify every disk write by re-reading the data.   This is the system’s very deliberate and very sensible design, but it is a crucial performance “gotcha.”

    SQLite is completely platform-independent and, believe it or not, it is in the public domain.   When used “properly,” as noted above, it provides excellent power and performance.   It’s easily the very best “server-free one-flat-file database” you’re gonna find anywhere (and therefore, everywhere).

Back to Seekers of Perl Wisdom