Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: fast simple DB (sqlite?) skeleton?

by WizardOfUz (Friar)
on Jan 25, 2010 at 17:24 UTC ( [id://819543]=note: print w/replies, xml ) Need Help??


in reply to fast simple DB (sqlite?) skeleton?

I have tried a version with DBI and SQLite, but insertion was very, very slow.

Have you read this?

Replies are listed 'Best First'.
Re^2: fast simple DB (sqlite?) skeleton?
by iaw4 (Monk) on Jan 25, 2010 at 17:57 UTC

    yes. I added a

    $dbh->do( "PRAGMA synchronous=OFF" );
    at the start (just after connecting), but it still ain't too fast. maybe I added it in the wrong spot, or something else went wrong?!?

    I should add that I can estimate how big my data base will be---is there a way to tell SQLite to preallocate?

    would dbm be better and faster in this sort of application? (I am on OSX.)

    /iaw

      This doesn't look too bad:

      #!/usr/bin/perl use strict; use warnings; use Benchmark ':hireswallclock'; use DBI (); my $DBH; setup_db(); Benchmark::timethis( 1, \&db_benchmark ); sub db_benchmark { my $sth = $DBH->prepare( 'INSERT INTO benchmark ( id ) VALUES ( ? +)' ); $DBH->begin_work; my $i = 0; while ( $i < 1_000_000 ) { $sth->execute( $i ); $i++; } $DBH->commit; return; } sub setup_db { $DBH = DBI->connect( "dbi:SQLite:dbname=benchmark.sqlite", "", "", { 'RaiseError' => 1 } ); if ( ! $DBH->tables( undef, undef, 'benchmark', 'TABLE' ) ) { $DBH->do( 'CREATE TABLE benchmark ( id INTEGER )' ); } return; }

      Results:

      #1 timethis 1: 10.4524 wallclock secs ( 9.35 usr + 0.16 sys = 9.51 C +PU) @ 0.11/s (n=1) #2 timethis 1: 9.9961 wallclock secs ( 9.31 usr + 0.18 sys = 9.49 CP +U) @ 0.11/s (n=1) #3 timethis 1: 9.85224 wallclock secs ( 9.29 usr + 0.17 sys = 9.46 C +PU) @ 0.11/s (n=1)

      System: AMD Athlon(tm) Dual Core Processor 4850e, HDD WD10EADS

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://819543]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (4)
As of 2024-04-24 02:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found