Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re^3: Best method to load a csv file.

by bart (Canon)
on Dec 18, 2004 at 12:01 UTC ( [id://415849]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Best method to load a csv file.
in thread Best method to load a csv file.

My experience with MS-Access leads me to believe that generating a new CSV file, in the format you want, is up to several orders of magnitude faster than inserting using DBI directly to insert every item into the database, one by one: a few seconds for generating the text files, as opposed to many minutes using DBI, for tables of a few ten thousands of records.

So, As a generic approach, I'd advice you to generate the flatfiles with perl, and use the database's loader/import mechanism to actually read the data into the database.

Note that reading all the data out of the database using Perl+DBI isn't nearly as slow as inserting/updating the records.

update I notice that jdtoronto, based on his own, independent experiences, came to the same conclusion (point 3).

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://415849]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2024-04-26 01:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found