in reply to SQLite and CPU Usage

There are several things you can try that should speed things up.

  1. Disable autocommit and commit your inserts in batches.

    See the DBI docs.

  2. Increase the number of pages of cache used using PRAGMA <default_>cache_size = nnnnn;

    The default is set to 2000. Increasing this to 10,000 seems to work well.

    See http://www.hwaci.com/sw/sqlite/lang.html#pragma

  3. Turn of the synchronous flag using PRAGMA <default_>synchronous = 0;.

    See the link above. Access the risks according the value of your data and the likelyhood of hardware failure on your system.


Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
"Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon