Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re^2: DBI Question: bulk insertion with Unique-key constraint

by lihao (Monk)
on Apr 24, 2008 at 20:45 UTC ( [id://682722]=note: print w/replies, xml ) Need Help??


in reply to Re: DBI Question: bulk insertion with Unique-key constraint
in thread DBI Question: bulk insertion with Unique-key constraint

But I need to pre-process some columns, otherwise some data got corrupted, like datetime format. So I guess I need at least to use Perl to go through the CSV file and adjust some columns like datetime format, save them into another file and then load data with MySQL. :)

Replies are listed 'Best First'.
Re^3: DBI Question: bulk insertion with Unique-key constraint
by perrin (Chancellor) on Apr 24, 2008 at 21:20 UTC
    Yes, doing a pass on the CSV file with perl and generating another CSV file to pass to MySQL is still probably faster than loading it row by row.

    By the way, INSERT IGNORE or INSERT ON DUPLICATE UPDATE are both typically faster than REPLACE if you do need to do it row by row.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://682722]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others scrutinizing the Monastery: (5)
As of 2024-03-29 08:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found