Re: Bulk data insertion into MySql

But your code does not use DBIx::BulkLoader::MySql at all! You are using straight DBI.

The fastest option for your bulk loading is probably using the LOAD DATA INFILE statement from MySQL.That statement expect a file with the data and does a very fast insert into the database. You can use Perl to transform your data into a format expected by the LOAD DATA INFILE statement.

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Comment on Re: Bulk data insertion into MySql

Replies are listed 'Best First'.
Re^2: Bulk data insertion into MySql by sowraaj (Novice) on Sep 03, 2011 at 12:38 UTC
Dear friend, When i use 'DBIx::BulkLoader::MySql' i got the error that shows @INC...., .pm file not find. I noted that it may due to http://code.activestate.com/ppm/DBIx-BulkLoader-Mysql/. if you don't mine explain the LOAD DATA INFILE concept in a small program(Now i insert 10,00,000rows/3min)I would like to reduce that time to seconds Thanks	[reply]
Re^3: Bulk data insertion into MySql by CountZero (Bishop) on Sep 03, 2011 at 19:57 UTC
I doubt that even the `LOAD DATA INFILE` statement will insert 10 lakh records in a few seconds. As far as DBIx::BulkLoader::Mysql is concerned, did you try installing it through `cpan`? CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James	[reply] [d/l] [select]