sroux has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks While discovering prepared statements with perl/dbi and sqlite I'like to optimize a piece of code I wrote as it is so slow...

The code below takes ages to insert some 6000 lines from a text file.

What could I easily improve?

Some guidance largely accepted as I'm not fully familiar with prepared statements yet.

Thanks for your guidance.

SebRoux

use DBI; my $dbfile = 'database.db'; # your database file my $dbh = DBI->connect( # connect to your database, create if "dbi:SQLite:dbname=$dbfile", # DSN: dbi, driver, database file "", # no user "", # no password { RaiseError => 1 }, # complain if something goes wrong ) or die "Cannot connect $DBI::errstr"; $dbh->do("DROP TABLE IF EXISTS log"); $dbh->do( "CREATE TABLE log ('date','time','server','application','database','us +er','msglevel','msgcode','msgcat','description')" ); #Parsing log $temp = "C:\\eclipse\\workspace\\Rightlog\\test.txt"; open( LOGFILE, "$temp" ) or die print "Error: could not open $temp\n"; $sth = $dbh->prepare("INSERT INTO log VALUES (?,?,?,?,?,?,?,?,?,?)"); while (<LOGFILE>) { chomp; my ( $date, $time, $server, $application, $database, $user, $msglevel, $msgcode, $msgcat, $description ) = split /\|/; $sth->execute( $date, $time, $server, $application, $database, $user, $msglevel, $msgcode, $msgcat, $description ); } close(LOGFILE); $dbh->commit; $dbh->disconnect();

Replies are listed 'Best First'.
Re: Optimize sqlite prepared statement
by Your Mother (Archbishop) on Aug 23, 2008 at 16:11 UTC

    The error is telling you how to fix it.

    (either destroy statement handles or call finish on them before disconnecting)

    Just add the "finish" line and the warning will go away-

    close(LOGFILE); $sth->finish();
Re: Optimize sqlite prepared statement
by sroux (Sexton) on Aug 23, 2008 at 12:52 UTC
    Well I changed a few things and it seems MUCH better now.

    But having an error... Added { RaiseError => 1, AutoCommit = 0 } But having the following message

    DBI::db=HASH(0x1a51adc)->disconnect invalidates 1 active statement han +dle (either destroy statement handles or call finish on them before d +isconnecting) at C:/eclipse/workspace/Rightlog/sqlite2.pl line 39.<p> closing dbh with active statement handles at C:/eclipse/workspace/Righ +tlog/sqlite2.pl line 39.