DBI placeholders less efficient?

rbuckner has asked for the wisdom of the Perl Monks concerning the following question:

I wrote a perl script that uses Perl DBI to connect and query an Oracle 9.2 database. I prepared a statement handle outside of a loop that is a select statement with 3 placeholder in the WHERE part of the statement. The statement will be executed approximately 800 times in the loop. e.g.

      my $sth = $dbh->prepare("select ... where fld1 = ? and
                                                fld2 = ? and       
                                                fld3 = ?");
      for ($i=0; $i<800; $i++0){
       ...
       $sth->execute($var2, $var2, $var4);
       ...
      }
[download]

Ran some test and it takes almost 2 min to execute the loop 4 times. I rewrote the script and prepared the statement inside the loop by concatenating a string with variable values. e.g.

     for ($i=0; $i<800; $i++){
       ...
      my $sql = "select ... where fld1=" . $var1 . 
                            " and fld2=" .$var2 .  
                            " and fld3=" . $var3;
      my $sth=$dbh->prepare($sql);
      $sth->execute();
       ...
     }
[download]

Same loop executes in less than 10 seconds. All the documentation I can find says the placeholder code should be more efficient and run faster than the concatenated code. Any idea what's going on?

Comment on DBI placeholders less efficient? Select or Download Code

Replies are listed 'Best First'.
Re: DBI placeholders less efficient? by dragonchild (Archbishop) on Mar 08, 2005 at 15:53 UTC
Placeholder code is primarily more maintainable and more secure, which is why it's used. It's more efficient if the server or the DBD can take advantage of not having to parse the same statement over and over. Oracle 9.2 definitely has this feature. You might want to consider doing the following (in no particular order) Turn on DBI's debug feature, which will tell you exactly what SQL is being executed. It looks like you're using integers or some other numeric type. You might need to explicitly tell DBD::Oracle with the bind_param() method exactly what type it is. That way it won't be quoting your number which does bad things to your optimizer. Being right, does not endow the right to be rude; politeness costs nothing. Being unknowing, is not the same as being stupid. Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence. Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.	[reply]
Re: DBI placeholders less efficient? by JediWizard (Deacon) on Mar 08, 2005 at 15:46 UTC
You might try adding `use DBD::Qracle qw(:ora_types);`. I have found in some curcumstances that has increased my efficency. You may also want to try using named place holders. I do not know what effect that will have on speed, but it may increase maintainability. also with named palce holders you can pass in a hash ref with `TYPE => ORA_INTEGER` or similar. A truely compassionate attitude towards other does not change, even if they behave negatively or hurt you —His Holiness, The Dalai Lama	[reply] [d/l] [select]
Re: DBI placeholders less efficient? by cfreak (Chaplain) on Mar 08, 2005 at 17:34 UTC
It doesn't look like you're using the value of $i in either of your loops, instead some arbitrary variables that were set elsewhere. If this is your real query then you don't need a loop at all, in fact your select statement would return the same row every time if you actually fetched from it. If you're trying to get 800 results then use one query with LIMIT If this is just an example then I think we need more info on what you're trying to accomplish. Lobster Aliens Are attacking the world!	[reply]
Re: DBI placeholders less efficient? by Anonymous Monk on Mar 08, 2005 at 17:12 UTC
Well, the first loop uses different variables than the second loop. Could it be that the query send is actually different? I'm quite surprised by the result, but it's just one test. A better test would be to use different values in the query. And repeat the test several times. You didn't by any chance run the first test, then the second test, and these two tests where the only times the query was actually run? If it took 1m and 59s to gather the data from disk to find the answer to the first time the query was run, and the rest of the time, the data was in the cache, it explains why the first one is slower.	[reply]
Re: DBI placeholders less efficient? by Thilosophy (Curate) on Mar 09, 2005 at 07:32 UTC
When using Oracle, you absolutely have to use placeholders. Anything else will completely kill your scalability. The only time I have ever had performance problems with placeholders is when this caused the execution plan (which indices to use and so on) for the query to become inefficient. This can happen if the data is very "skewed": If you use placeholders, Oracle cannot take the actual values of these columns into consideration and has to assume "average" data points. If you have, for example, a table "employees" and 95% of them are male, and then have a query for only female employees, not using a placeholder for the gender might actually be beneficial. Another example would be a limited query (return just the first x rows) which can possibly create a better plan if "x" is known at preparation time. However, this should happen only in very rare cases. If you can consistently reproduce this problem, contact your DBA to find out what execution plans your query creates.	[reply]