in reply to MySQL Select Speed Optimisation
The "OR" operator will slow down your query mercilessly. However, using the "IN" operator, MySQL will use the primary key (or any associated index) and run very fast.
SELECT * FROM some_table WHERE key IN ( ?, ?, ?);
An alternative to using placeholders is to prepare this statement using a join.
my @searchkeys = qw(a b c d); my $query = qq{ SELECT * FROM some_table WHERE key IN ( } . join(",", map( {$dbh->quote($_)} @searchkeys)) . qq{)};
update (1). See also Using OR in SELECT statments safely.
update (2). Some benchmarking, to show that the IN operator is truly faster. I ran the query against a large table containing 500,000 records.
#!/usr/bin/perl -w use DBI; use strict; use Benchmark; my $dbh = DBI->connect("DBI:mysql:chess_db;host=localhost" . ";mysql_read_default_file=$ENV{HOME}/.my.cnf", undef, undef, {RaiseError => 1}) or die "can't connect\n"; my $query = qq{SELECT count(*) from PGN_base_data where }; my @searchkeys = ((100_001 .. 100_800)); my $or_query = $query . join( " OR " , map( { "ID = $_"} @searchkeys)); my $in_query = $query . "ID IN (" . join( "," , @searchkeys) . ")"; my $sth_or = $dbh->prepare($or_query); my $sth_in = $dbh->prepare($in_query); sub get_it { my $sth = shift; $sth->execute; my $result = $sth->fetchrow_arrayref; $sth->finish; #print $result->[0], "\n";; } timethese (2000, { 'or' => sub {get_it $sth_or}, 'in' => sub {get_it $sth_in} }); $dbh->disconnect(); __END__ with 400 keys in the query Benchmark: timing 4000 iterations of in, or... in: 17 wallclock secs ( 0.37 usr + 0.09 sys = 0.46 CPU) or: 52 wallclock secs ( 0.70 usr + 0.08 sys = 0.78 CPU) with 800 keys in the query Benchmark: timing 2000 iterations of in, or... in: 17 wallclock secs ( 0.30 usr + 0.10 sys = 0.40 CPU) or: 95 wallclock secs ( 0.41 usr + 0.06 sys = 0.47 CPU)
Note. There is no possible comparison with UNION, because it would return several rows, which should be added up in the client.
My test query is only asking for a COUNT(*), thus returning only one row.
_ _ _ _ (_|| | |(_|>< _|
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re: Re: MySQL Select Speed Optimisation
by tachyon (Chancellor) on Mar 27, 2003 at 10:58 UTC | |
by IlyaM (Parson) on Mar 27, 2003 at 11:36 UTC | |
by VSarkiss (Monsignor) on Mar 27, 2003 at 15:59 UTC |