gojira has asked for the wisdom of the Perl Monks concerning the following question:

I am programming perl for a while now but mysql, and the combination of perl with mysql is very new to me. I wrote a perl script that is supposed to do kmedoids-clustering of several thousand promoters (human genome).

Many instances of this script are running on a Grid Engine cluster. When I run this script with a random selection of a few hundred promoters only, everything works fine.

But now I tried to cluster the full dataset of several thousand promoters and get this error:

"DBD::mysql::st fetchrow_array failed: fetch() without execute() at chipchip.pl line 144, <L3> line 14446. "

Here is the part of my script where it happens, I think:

foreach $item (@{$clusters}) { $ID = $units[$i][0]; $query = "select * from finalLevel3 where id=\"$ID\";"; $sth = $dbh -> prepare($query); $sth -> execute; @row = $sth -> fetchrow_array; # convert the mysql result back to a string again $promoter_line = join "\t", @row; $promoter_line .= "\n"; $parser_hash{$item} .= $promoter_line; $i++; }

I found a thread in this forum which I think is a very similar problem and the solution, however I must say that I simply don't understand how to solve this and what's going on. That thread is node 729564. I'm happy that this problem seems already to be solved but I simply don't understand the solution - a more verbose explanation would be very, very much appreciated.

Do I need two separate $dbh - handles? If yes do I need to connect them separatedly?

I would be very thankful if someone could explain to me how to fix this. Also, the thing is that this error only shows up if I do the clustering on the whole dataset (2 days runtime on Grid Engine), so it would be superduper cool if I could get it right the first or second time because it takes so long to reproduce the error. Thank you.

Replies are listed 'Best First'.
Re: perl, mysql: "fetchrow_array failed: fetch() without execute()"
by ikegami (Patriarch) on Dec 29, 2008 at 06:13 UTC

    If it's the same problem as the other thread, proper use of my will fix it. Don't you use use strict; use warnings;?

    my $ID = ...; my $query = ...; my $sth = $dbh -> prepare($query); $sth -> execute; my @row = $sth -> fetchrow_array;

    Another possibility is that there was an error performing the execute. Since you don't check for errors, did you specify RaiseError => 1?

    Two other comments:

    • my $i = 0; for my $item (@{$clusters}) { ... $i++ }

      can be written more simply as

      for my $i (0..$#$clusters) { ... }
    • This one is a serious problem. Just adding quotes around the contents of $ID doesn't properly convert it from arbitrary text into an SQL string literal. You need one of the following, preferably the second:

      for my $i (0..$#$clusters) { my $ID = $units[$i][0]; my $query = "select * from finalLevel3 where id=" . $dbh->quote($I +D); my $sth = $dbh -> prepare($query); $sth -> execute; my @row = $sth -> fetchrow_array; ... }
      my $query = "select * from finalLevel3 where id=?"; my $sth = $dbh -> prepare($query); for my $i (0..$#$clusters) { $sth -> execute( $units[$i][0] ) my @row = $sth -> fetchrow_array; ... }
      This sequence of statements can be easier:
      $query = "select * from finalLevel3 where id=\"$ID\";"; $sth = $dbh -> prepare($query); # I would suggest this: my $query = $dbh->prepare("SELECT * FROM finalLevel3 WHERE id='$ID'");

      I personally like to captialize the SQL key words, but preferences vary. Also the single quote within the double quote will expand $ID just fine. There is no need for \" in this situation. I was amazed when I learned this.

      I am also a bit confused about this:

      foreach $item (@{$clusters}) { #..... }

      I would think that since you are iterating over clusters, that something like this would be more clear:

      foreach my $cluster (@$clusters) { #..... }
      So when do you need the extra {}? You need this when dereferencing an indexed expression, but not a non-indexed one. You also need the extra {} to make a list from a list ref returning function.
      my @list = @{listRefReturningFunction()}; #also consider: my $r = {"a" => [1,2,3,4], "b" => [5,6,7,8], }; print @{$r->{"a"}}; #prints 1234 #here print @$r->{"a"} won't work #because this is a subscripted expression # now with list... my @x =(1,2,3,4); my $xref=\@x; print @$xref; #prints 1234 also #this works because there is no subscript

        Did you really mean to reply to my post? It seems you ignored everything I said.

        This sequence of statements can be easier:
        my $query = $dbh->prepare("SELECT * FROM finalLevel3 WHERE id='$ID'");

        Just adding quotes around the contents of $ID doesn't properly convert it from arbitrary text into an SQL string literal. It's an bug or injection attack waiting to happen. It's also very inefficient. You're compiling the query over and over and over again. The following is a whole world better:

        # Outside of loop my $sth = $dbh->prepare("SELECT * FROM finalLevel3 WHERE id=?"); $sth->execute($ID);

        I would think that since you are iterating over clusters, that something like this would be more clear:

        If you look closely, you'll notice he's actually iterating over the cluster indexes, so the following would be even better:

        for my $i (0..$#$clusters)

        So when do you need the extra {}? You need this when dereferencing an indexed expression, but not a non-indexed one.

        @{ foo() } would also need it. The correct answer would be "Pretty much anything other than a variable name" or maybe even "Everything other than a variable name".

Re: perl, mysql: "fetchrow_array failed: fetch() without execute()"
by runrig (Abbot) on Dec 29, 2008 at 06:11 UTC
    No, you don't need two statement or database handles. Do you know what the $ID is when the statement fails? You're iterating over @$clusters but grabbing $ID's from @units. Maybe $ID is undef? I'd use placeholders something like this (with maybe some extra error handling in case nothing is selected into @row...which would be easy, just remove the "1" from the code below):
    my $sth = $dbh->prepare("select * from finalLevel3 where id=?"); foreach $item (@{$clusters}) { $ID = $units[$i][0]; unless (eval { @row = $dbh->selectrow_array($sth, undef, $ID); 1 }) { die "Error on ID [$ID]: $@"; } # convert the mysql result back to a string again $promoter_line = join "\t", @row; $promoter_line .= "\n"; $parser_hash{$item} .= $promoter_line; $i++; }