in reply to copying mysql table data to oracle table
A few comments
I would change "getOracleRowHahs" to
and the call tosub getOracleRowHash{ my $odbh = shift; my $osth = $odbh->prepare("select pk from emp"); $osth->execute(); my $tableref; while(my $row = $osth->fetchrow_hashref()){ no warnings; $tableref->{$row->{'pk'}} = $row; } return $tableref; }
No need to connect twice and the use of the & is no longer suggested. Just request the one var since that is all you want now. Changing it to a array fetch may speed it up significantly, but that is left as a exercise. Also note the case change to follow your table definition.my $oracleTableHash = getOracleRowHash($odbh);
In the while(my @row = $ssth->fetchrow_array()){ loop you re-prepare your insert every time. That is going to cost you. Also after a DBA changed the native variable order on me i always include variable names for all but the most trivial tasks.
Note too the begin_work/commit calls that will speed up the insert a WHOLE BUNCH!!! The commit_ct test is to limit the size of the internal "holding" list that begin_work creates;my @vnames=qw/ pk File_name Cl_file_size_in_bytes Cl_checksum CL_created_date_time Status Mn_file_size_in_bytes Mn_checksum Failed_retries Processed_date_time ftp_acknowledgement ohf_processing_status ohf_processing_datetime ohf_total_rows ohf_failed_rows ohf_processed_rows hdfs_processing_status hdfs_processing_datetime hdfs_checksum hdfs_file_size_in_bytes /; my $vlist=join(',',@vnames); my $table='cloverleaf_ingestion'; my @qs=map{'?'} @vnames; my $q=join(',',@q); my $query = "select $vlist from $table"; my $ssth = $sdbh->prepare($query) || die "Error in prepare t +o mysql: $DBI::errstr\n";; my $insertquery = "insert into $table($vnames) values($q)"; my $osth = $odbh->prepare($insertquery) || die "Error in prepare: +$DBI::errstr"; $osth->begin_work; $ssth->execute() || die "Error in execute to mysql: $DBI::er +rstr\n";; my $commit_ct =500; my $inserts=0; while(my @row = $ssth->fetchrow_array()){ if(!(exists($oracleTableHash->{$row[0]}))){ $inserts++; if ($inserts>$commit_ct){ $inserts=1; $osth->commit or die 'something'; $osth->begin_work; } $osth->execute(@row); } } $osth->commit or die 'something';
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: copying mysql table data to oracle table
by CSharma (Sexton) on Aug 24, 2017 at 12:41 UTC | |
by huck (Prior) on Aug 24, 2017 at 13:11 UTC | |
by CSharma (Sexton) on Aug 25, 2017 at 07:52 UTC |