in reply to Re^2: How to improve MYSQL search performance of perl?
in thread How to improve MYSQL search performance of perl?

I would take trammell's suggestion a step further and also not recreate the statement handle every time and actually take advantage of the statement handle (and placeholders)-- i think this will have a decent improvement in the performance (amount of gain is probably db-dependent): 

my $dbh = DBI->connect('DBI:mysql:diet', {RaiseError => 1, AutoCommit 
+=> 0} ) || die "Failed to connect: $DBI::errstr";
my $sth = $dbh->prepare( qq{select topic FROM table1 WHERE uri LIKE ?}
+ );

search($sth, 'foo');
search($sth, 'bar');

$sth->finish();
$dbh->disconnect(); #disconnect from database;

sub search{
  my $sth = shift;  # require statement handle (this could probably be
+ a global var instead if desired)
  my $q = shift;    # take search parameter from html <form/>
  my $found = 0; #initialize category found count;
  $sth->execute($q);
  my $rows = $sth->fetchall_arrayref( {} );
  printf "%d rows found for '%s'.\n", scalar(@$rows), $q;
  foreach my $row (@$rows){
    printf "   Topic: %s\n", &topic($row->{topic});
  }
}

[download]

Replies are listed 'Best First'.
Re^4: How to improve MYSQL search performance of perl?
by nan (Novice) on Aug 24, 2005 at 17:06 UTC

    Hi,

    Many thanks for the code as I learned a lot from you. But even opened database once, the search is still slow (running 200 queries one by one) and I always recieve Internal Server 500 Error after a long waiting and can't see any results back to screen (before modifying, I could see a few results back as the database open and close everytime). I'm not sure it is due to the configuration of apache server or MySQL database so I'm rebuilding the database now to see if it helps.

    thanks again,

    Nan

[reply]
      Can you post your revised code? Are you re-using the statement handle? What's the error in the error_log (or if you run on command line)?

      Also, try adding $|=1; in your script to make sure it's not buffereing so yuo'll see output as it's happening.

      If the loop to execute the queries is good now (resuing handles, etc), then this probably just goes back to the issue mentioned elsewhere in this thread about SQL performance .. the speed will depend entirely on your SQL, table schema, and indexes (can you provide any of those?).
[reply]
[d/l]

        Hi,

        After reading through Chapter 10 in MySQL Reference Manual, I'm trying to rebuild the table and index and I'll let you know what happens after that.

        Actually, the error log just says "personal.pl: Unrecognized escape \\h passed through at F:/httpserv/Top/cgi-bin/personal.pl line 11., referer: http://localhost/form.html" but the line 18 is about database connection:

        my $dbh = DBI->connect('DBI:mysql:diet', $ref) || die "Failed to connect: $DBI::errstr";

        And it's weird that the script works fine without using Apache server...

        Many thanks,

        Nan

[reply]

In Section Seekers of Perl Wisdom