You are preparing the same statement over and over. Use placeholders instead.

If you pass DBI the RaiseError flag on connecting, you can avoid checking the success of each DBI operation manually.

You also read the entire file into an array, then process it element by element. Read as you go instead.

While we're at it, don't open the file yourself at all. Using the diamond operator to read from the files in @ARGV will handle errors automatically and lets you handle multiple files at no extract cost.

There's also no provision in your script to pass the connection parameters in from the outside, so you have to change the source for every database it is supposed to run against. Look into Getopt::Long.

#!/usr/bin/perl -w use strict; use DBI; use Text::CSV; use Getopt::Long; use constant USAGE => "usage: $0 [ -h hostname ] [ -p port ] --db dbna +me -u user -p passwd -t table file.csv [ file.csv ... ]"; GetOptions( 'h|host=s' => \( my $opt_host = "localhost" ), 'p|port=i' => \( my $opt_port = "3306" ), 'db=s' => \( my $opt_dbname ), 'u|user=s' => \( my $opt_user ), 'p|pass=s' => \( my $opt_pass ), 't|table=s' => \( my $opt_table ), ) or die USAGE; die USAGE if not( $opt_dbname and $opt_user and $opt_pass and $opt_table ); my $dbh = DBI->connect( "dbi:mysql:dbname=$opt_dbname;host=$opt_host;port=$opt_port", $opt_user, $opt_pass, { RaiseError => 1, PrintError => 0 } ) or die "Connection error: $DBI::errstr"; my $csv = Text::CSV->new(); my $sth = do { $csv->parse( scalar <> ); # first line my @field = $csv->fields(); my $sql = "INSERT INTO $opt_table (" . join( ',', @field ) . ") VALUES (" . join( ',', ( '?' ) x @field ) . ")"; print STDERR "Using query: $sql\n"; $dbh->prepare( $sql ); }; # # broken # while( <> ) { # $csv->parse( $_ ) or next; # $sth->execute( $csv->fields ); # } my ( $buff, $lnr ); while( <> ) { $buff .= $_; if( $csv->parse( $buff ) ) { $sth->execute( $csv->fields ); $buff = ''; $lnr = $. + 1; } } die "Error on line $lnr\n" if length $buff;

Untested.

Update: fixed code as per tilly's example.

Makeshifts last the longest.


In reply to Re: Load CSV file into Database by Aristotle
in thread Load CSV file into Database by fauria

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.