in reply to Loading bulk data into SQLite

hangon,
If you are doing bulk loading from a text file, it is a bit complicated. SQLite 2.x used to have the COPY command - you can see where I used it in Fastest Rising Monks - Revisited (old ugly code - but look at Build_DB()). It disappeared in 3.x but there are a number of ways to accomplish the same thing. See this for instance (search for .import). You will need to do this through the SQLite shell. If memory serves, I once did this using DBI::Shell.

Cheers - L~R

Replies are listed 'Best First'.
Re^2: Loading bulk data into SQLite
by hangon (Deacon) on Nov 30, 2008 at 03:41 UTC

    Thanks L~R. For some reason I was fixated on "loading" and missed the ".import" command. I installed a standalone version of SQLite to try out its command shell. Still had to process the file through Perl to get it into a suitable format, but afterwards loading it using .import was extremely fast.

[reply]
      hangon,
      Still had to process the file through Perl to get it into a suitable format

      I forgot to mention that the .mode command can switch to various different formats such as tabs a la 

      .mode tabs;
.import <file> <table>;

      [download]

      Cheers - L~R

[reply]
[d/l]

In Section Meditations