in reply to Re^4: DBD::Pg copy issues
in thread DBD::Pg copy issues

$dbh->func( 'COPY snapshot_$formName ( name, value, collect_date, omp_
+id ) FROM STDIN',
            'putline' );

[download]

There are three problems with preceding line.

  1. The first and most serious is that you used single quotes but appeared to expect $formName to interpolate into the string. Single quotes prevent this. You probably meant to use double quotes.
  2. The second is that 'value' is just a really poor name for a value. Its like calling a variable $variable. Use descriptive names for variables and table attributes. Its part of writing maintainable code.
  3. The 'name' name is typically a reserved word in SQL and you'll find that a lot of things play badly with schemas that use attributes of this name. I suggest you find something more appropriate like "collection_name."

    my $count = 0;
for ( ... ) {
    ... $names[ $count ]

    $count++;
}

    [download]

    It looks like somehow your indexes in @values are synchronized with the indexes in some @names list. When you manage the current index manually, you make it likely that it will be managed incorrectly. Let perl do this for you.

    foreach my $ix ( 0 .. $#names )
    my $value = $values[ $ix ];
    if ( ! $names[ $ix ] ) {
    }
}

    [download]

    Now here's what I think a reasonable re-write of your code would look like. I didn't change the table attribute name so that's still present in its likely-buggy form.

    $formName = ...;
@form_out = ...;
$ga = { ... };
$collect_date = ...;
$omp_id = ...;
my @value_names = split ' ', $ga->{ 'outputString'}{ $formName };
chomp @value_names;
my $table_name = "snapshot_$formName";

$dbh->func( "COPY $table_name ( name, value, collect_date, omp_id ) FR
+OM STDIN",
            "putline" );
print "Started a line.\n";

for my $packed_values ( @form_out )
{
    my @values = split /:/, $packed_values;
    my @values_to_use_ix = grep $value_names[ $_ ],
                           0 .. $#value_names;
    for my $ix ( @values_to_use_ix )
    {
        my $value = $values[ $ix ];
        my $value_name = $value_names[ $ix ];

        $value = 'NULL' if not defined $value;
        $value =~ s/^\s+//;
        $value =~ s/\s+$//;
        # The following line will remove any internal tabs which would
+ mess with
        # the column count.
        $value =~ s/\s+/ /g;
        $value = 'NULL' if not length $value;

        my $row = join( "\t",
                        $value_name,
                        $value,
                        $collect_date,
                        $omp_id );

        eval { $dbh->func( $row, 'putline' ) }
            or die "Couldn't write `$row' to $table_name: $@, " . $dbh
+->errstr;
    }
}

    [download]

Replies are listed 'Best First'.
Re^6: DBD::Pg copy issues
by tradez (Pilgrim) on Sep 14, 2004 at 19:42 UTC
    Alright this is killling me now. Here is my new code and thank you so much for your help. This thing can't even get passed the first COPY statement without dying, and I can't get any type of error message back other then it failed! 
    
$dbh->func("COPY snapshot_$formName (name, value, collect_date, omp_id
+) from STDIN", "putline"
        or die "Couldn't write COPY STATEMENT to snapshot_$formName: $
+@, " . $dbh->errstr;
    print "Started a line\n";

    foreach my $values (@form_out){
      #my $cmd = `rm -f $form_out`;
      my @values = split(/;/, $values);
      my @names = split(/ /, $ga->{'outputString'}{$formName});
      my $count = 0;
      foreach my $ix (0 .. $#names ){
        my $value = $values[$ix];
        if (!$names[$count]){
         print "$value element # $count does not exist in array\n";
        }
        my $name = $names[$count];
        $value = 'NULL' if not defined $value;
        $value =~ s/^\s+//;
        $value =~ s/\s+$//;
        # The following line will remove any internal tabs which would
+ mess with
        # the column count.
        $value =~ s/\s+/ /g;
        $value = 'Null' if not length $value;
        my $row = join("\t",
                       $name,
                       $value,
                       $date,
                       $ompId);
        $dbh->func($row , 'putline' )
        or die "Couldn't write `$row' to snapshot_$formName: $@, " . $
+dbh->errstr;
         $count++;
      }
    }
    $dbh->func( "\\.\n", 'putline' );
    print "ended that line\n";
  }

    [download]


    Tradez
    "Never underestimate the predictability of stupidity"
    - Bullet Tooth Tony, Snatch (2001)
[reply]
[d/l]
      Oh that's too bad! I would have just been using the psql binary directly by now. If you really want to do this without the external process you should probabyl raise this problem on one of the PostgreSQL mailing lists.
[reply]
        oh my god!!!! arghh, well I would have given up and tried this before but I am just too competitive. But now, arghh, i get this! 
        DBD::Pg::db do failed: ERROR:  missing data for column "collect_date" 
+at ./getSnapshot.pl line 175, <FORM_OUT> line 102.
Couldn't write COPY STATEMENT to snapshot_btseqp: , ERROR:  missing da
+ta for column "collect_date" at ./getSnapshot.pl line 175, <FORM_OUT>
+ line 102.

        [download]
        when i do this: 
        my $row = join("\t",
                       $name,
                       $value,
                       $date,
                       $ompId);
print INPUTFILE "$row\n";
 }
    }

    print INPUTFILE "\\.\n";
    $dbh->do("COPY snapshot_$formName (name, value, collect_date, omp_
+id) from '$db_input'")
        or die "Couldn't write COPY STATEMENT to snapshot_$formName: $
+@, " . $dbh->errstr;

        [download]
        Seriously, what the hell. Also, when I do the same copy statement from the cmd line, it works. I hate this. I even catch to make sure that $date, which is always there is there and it is.


        Tradez
        "Never underestimate the predictability of stupidity"
        - Bullet Tooth Tony, Snatch (2001)
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom