in reply to Better way to work with large TSV files?

Convert your file of data into a file of SQL statements. Then, run the SQL statements in a transaction using your RDBMS's standard client (which will be loads quicker than DBI).

Of course, you could dispense with DBI altogether and use your RDBMS's loader. Oracle and MySQL both support most xSV formats (such as TSV, CSV, PSV, etc.)

------
We are the carpenters and bricklayers of the Information Age.

Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose

I shouldn't have to say this, but any code, unless otherwise stated, is untested

  • Comment on Re: Better way to work with large TSV files?

In Section Seekers of Perl Wisdom