in reply to DBI: speed up insertions in generating database

If you perl to parse data (of course if you prefer/need divide it) until you obtain a unique csv file and after (assuming file is mydata.csv)
LOAD DATA LOCAL INFILE 'mydata.csv' INTO TABLE mytable FIELDS TERMINATED BY ',' ENCLOSED BY '"' (list, of , your , table, field);