trying to execute "LOAD DATA INFILE" using DBI::Mysql

jjohhn has asked for the wisdom of the Perl Monks concerning the following question:

Is there a way to execute a "LOAD DATA INFILE" MySQL statment using perl? I am trying to more or less duplicate the functionality of Oracle's sqlldr, which lets me load data from a text file into a table from the commandline. MySQL requires that one loads text data from a mysql> prompt. Is there any time saving to using a "LOAD DATA INFILE" vs just parsing the file in perl and executing a series of INSERT statements. For some reason I thing that LOAD DATA INFILE is faster, but I am not sure.

Comment on trying to execute "LOAD DATA INFILE" using DBI::Mysql

Replies are listed 'Best First'.
Re: trying to execute "LOAD DATA INFILE" using DBI::Mysql by perrin (Chancellor) on May 16, 2007 at 19:41 UTC
Yes, you can use LOAD DATA INFILE the same as you would any other SQL statement from perl. It is much faster than any other way of loading data.	[reply]
Re: trying to execute "LOAD DATA INFILE" using DBI::Mysql by bradcathey (Prior) on May 17, 2007 at 00:55 UTC
Here's some code that we use all day long to allow clients to refresh a database themself. Note: we have strong code ahead of this that validates the MIME type and file extension of their upload, and then code that checks the number of columns and names in the CSV file to match the DB table. `my $dbh = $self->dbconnect(server =>'master', db => 'admin'); my $stmt = qq~TRUNCATE TABLE $table~; #dump the old $dbh->do($stmt); $stmt = qq~LOAD DATA LOCAL INFILE ? INTO TABLE $table FIELDS TERMINATED BY "," LINES TERMINATED BY "\r\n" IGNORE 1 LINES~; my @bind = ("$path/$new_file_to_upload"); $dbh->do($stmt, undef, @bind); unlink "$path/$new_file_to_upload";` [download] Watch the `FIELDS/LINES TERMINATED BY...` stuff. Good luck! —Brad "The important work of moving the world forward does not wait to be done by perfect men." George Eliot	[reply] [d/l] [select]
Re: trying to execute "LOAD DATA INFILE" using DBI::Mysql by j1n3l0 (Friar) on May 17, 2007 at 10:03 UTC
Yes. You can use the mysqlimport option from a perl system call like so `#!/usr/bin/perl -w use strict; my $command = qq(mysqlimport -u $username -p$password -L $database $in +file); system($command) == 0 or die qq(system $command failed: $?); exit;` [download] You should get output that looks like this: `databasename.tablename: Records: 1 Deleted: 0 Skipped: 0 Warnings: +0` [download]	[reply] [d/l] [select]