in reply to DBI: speed up insertions in generating database
One thing that may help is to prepare all your insert statements beforehand. Something like this:
This may or may not help, depending on lots and lots of factors.my %prepares = ( table1 => $dbh->prepare("insert table1 (...) values (?, ...)", table2 => $dbh->prepare("insert table2 (...) values (?, ...)", # and so on ); # Now process the file open FILE, '<', $wherever; while (<FILE>) { # set $table and @values from $_ $prepares{$table}->execute(@values); }
As for using CSV files, you're probably thinking of using mysql loader. That may be a win, again depending on lots of factors. Essentially, instead of executing the insert statement in your program, you'd write the data to one of several files (one for each table), then use loader to populate one table at a time.
It's very hard to say whether any of these will be a win. The factors to consider include the amount of data, the number of tables, how they're indexed, whether you're doing this just once or on a regular basis, and so on.
|
|---|