I have tried two different approaches. One is to load the entire file into a list, prepare the INSERT statement, and then $sth->execute(@list). This is reasonably fast, and works. But, if Win32 (I have no other choice here) crashes during a run, I'm left with a partially-inserted table. Not OK. I'm aware of transactions, but given the size I don't want to have a 2GB file (many are this large) fail near the end and lose a couple hours' worth of statements.

The second approach, which is more fault tolerant, is to use cpan://Tie::File to tie the source file to an array, then pop the array after successfully inserting a row. Like this:

$sth->prepare("INSERT INTO Blah (a,b,c) VALUES (?,?,?)");
tie @list, 'Tie::File', $filename;
while (@list) {
  my @insert = processRow($list[$#list]); #this splits the records int
+o an array and does some data cleaning
  $sth->execute(@insert);
  #And check for errors and all that...
  pop @list;
}

untie @list;
unless ($errors) { unlink $filename; }
[download]

This is very reliable, and allows me to run more than one instance (as long as I work on separate files -- I am using flock). However, it's slow as a dehydrated camel.

Is there a Better Way to accomplish this?

Update:Aristotle has suggested replacing Tie::File with File::ReadBackwards. The latter module has a subset of functionality that I'm using, and is much faster than the former for what I'm doing. Thank you Aristotle!

$sth->prepare("INSERT INTO Blah (a,b,c) VALUES (?,?,?)");
tie *TSV, 'File::ReadBackwards', $filename;
while (<TSV>) {
  my @insert = processRow($_); #this splits the records into an array 
+and does some data cleaning
  $sth->execute(@insert);
  #And check for errors and all that...
  truncate $filename, tell TSV; #pops the last line
}

untie *TSV;
unless ($errors) { unlink $filename; }
[download]

Thanks to all the Monks for your wisdom.