In modern Perls -- I'm not sure which versions qualify here, maybe 5.6+ -- Perl will check whether the contents of the variable has changed. If the content of the variable has not changed, the regexp is not recompiled.

For example, compare

my @words = (
   'foo',
   'bar',
);

foreach (@array) {
   foreach my $word (@words) {
      if (/\Q$word/) {  # BAD!! $word always changes.
         print;
         last;
      }
   }
}
[download]

to

my @words = (
   'foo',
   'bar',
);

foreach my $word (@words) {
   foreach (@array) {
      if (/\Q$word/) {  # GOOD!! regexp only recompiled when needed.
         print;
         last;
      }
   }
}
[download]

It's not always practical to change the order of the loops. For example, when one of them reads from a file. In that case, the solution is to precompile the regexps. For example, compare

my @words = (
   'foo',
   'bar',
);

while (<FILE>) {
   foreach my $word (@words) {
      if (/\Q$word/) {  # BAD!! $word always changes.
         print;
         last;
      }
   }
}
[download]

to

my @words = (
   'foo',
   'bar',
);

# Precompile the regexps.
my @regexps = map { qr/\Q$_/ } @words;

while (<FILE>) {
   foreach my $regexp (@regexps) {
      if (/$regexps/) {     # GOOD!! $regexp is a compiled regexp.
     #if ($_ =~ $regexp) {  # GOOD!! Alternate syntax.
         print;
         last;
      }
   }
}
[download]

If you're trying to match constant strings rather than regexps, then I recommend Regexp::List:

use Regexp::List ();

my @words = (
   'foo',
   'bar',
);

my $regexp = Regexp::List->new()->list2re(@words);

while (<FILE>) {
   print if /$regexp/;
  #print if $_ =~ $regexp;  # Alternate syntax.
}
[download]

By the way,
for ($loop_index = 0; $loop_index < $#patterns; $loop_index++) {
is much less readable and no more efficient than
for my $loop_index (0..$#patterns) {
You could also have used
foreach (@patterns) {

Finally, in your case, I'd use

my @patterns = qw(
   create
   drop
   delete
   update
   insert
);

my $regexp;
$regexp = Regexp::List->new(modifiers => 'i')->list2re(@patterns);
$regexp = qr/\b(?:$regexp)\b/;

...

while ($data = $sth->fetchrow_arrayref()) {
   # index is faster than regexps on constant strings.
   next if index(lc($data->[10]), 'tempdb') >= 0;

   if ($data->[13] =~ $regexp) {
      print "$data->[3] $data->[9] $data->[10] $data->[13]\n";
      last;
   }
}
[download]

Update: Bug Fix: Changed $word to $_ in map's code block.

In modern Perls -- I'm not sure which versions qualify here, maybe 5.6+ -- Perl will check whether the contents of the variable has changed. If the content of the variable has not changed, the regexp is not recompiled.
...

   foreach my $word (@words) {
      if (/\Q$word/) {  # BAD!! $word always changes.
[download]

Hi ikegami,

Thanks for the very informative post! A quick question: I thought that in a loop like the one above, the iterator variable ($word) was temporarily aliased to each value of the array. So I would expect that as long as the array's contents didn't change, perl would know not to recompile the RE, and so both of your above examples would be the same speed.

But a quick Benchmark agrees with you: putting the RE in the outer loop is about twice as fast as in the inner loop.

Any hints as to what's wrong with my understanding of variable aliasing or RE caching?

Thanks!

What matters is the value of $word, because we're using the value of $word to compile the regexp. While the value of the variable to which $word is/was aliased doesn't change, the value of $word itself does change.

In the first pass, $word is "foo". The regexp was thus compiled with to /foo/. In the second pass, $word is "bar". We obviously need to recompile the regexp because we want /bar/ and it's currently /foo/. Whether $words[0] and $words[1] changed or not is completely irrelevant.

You might be thinking that the compiled regexp is stored with the variable used in the regexp. It's not. That wouldn't work when no variables or multiple variables are used to create the regexps. Instead, the uncompiled regexp or the values of the variables used to compile the regexp -- I don't know which -- is stored along with the compiled regexp in the code.

Every once in a while, I see a post like this and wish I could ++ it more than once. For newcomers to a language, seeing "badform/goodform" examples is really a great way to understand the benefits of different approaches.

--
[ e d @ h a l l e y . c c ]

That depends on a whole lot of factors. In old perls, that was true; modern perls have optimisations to avoid recompiling patterns if the interpolated variables haven’t changed.

However, note that a match such as $foo =~ /$bar/ is magical if $bar is a pattern precompiled with qr, in that the pattern compilation phase is skipped completely. (This applies when the pattern consists of nothing but the interpolated variable.)

And finally, the (?{}) delayed evaluation construct lets you embed pre-compiled patterns in another pattern without recompiling the embedded pattern.

Makeshifts last the longest.

modern perls have optimisations to avoid recompiling patterns if the interpolated variables haven’t changed

Modern perls are even smarter than that. They look at the string after interpolation, and if that's the same, the regex isn't executed:

$ cat xx
#!/usr/bin/perl

use strict;
use warnings;

foreach my $x (["foo", "bar"], ["fo", "obar"]) {
    my ($foo, $bar) = @$x;
    "" =~ /$foo/;
    "" =~ /$foo$bar/;
}
 
__END__
$ perl -Dr xx 2>&1 | 
  perl -nle 'if (/EXECUTING/ .. eof) {print if/^Compiling/}'
Compiling REx `foo'
Compiling REx `foobar'
Compiling REx `fo'
[download]

Perl won't recompile the second regex, since while the values of both $foo and $bar have changed, the value of "$foo$bar" hasn't.
Perl --((8:>*