Doing a great big regex with lots of /this|that|the other|etc/ is documented as probably being slower than /this/ || /that/ || /the other/ || /etc/ (or at least was documented -- this may have been dropped when perlre.pod was created, perhaps because it is no longer true). Of course, this will vary by case so a benchmark will tell you for sure.

So, based on my old bias, I'd code it one of these two ways:

@problem = (
"0  OBS",
"AT LEAST",
"EXTRANEOUS",
"CARTESIAN",
"CLOSING",
"CONVERT",
"DIVISION BY ZERO",
"DOES NOT EXIST",
"DUE TO LOOPING",
"END OF MACRO",
"ENDING EXECUTION",
"ERROR",
"ERRORABEND",
"ERRORCHECK=STRICT",
"EXCEED",
"HANGING",
"HAS 0 OBSERVATIONS",
"ILLEGAL",
"INCOMPLETE",
"INVALID",
"LOST CARD",
"MATHEMAT",
"MERGE STATEMENT",
"MISSING",
"MULTIPLE",
"NOT FOUND",
"NOT RESOLVED",
"OBS=0",
"REFERENCE",
"REPEAT",
"SAS CAMPUS DRIVE",
"SAS SET OPTION OBS=0",
"SAS WENT",
"SHIFTED",
"STOP",
"TOO SMALL",
"UNBALANCED",
"UNCLOSED",
"UNINITIALIZED",
"UNREF",
"UNRESOLVED",
"WARNING"
);


# First way:
my $code= "sub { /("
  . join ")/i || /(", map {"\Q$_\E"} @problem;
$code .= ")/i }";
my $match= eval $code;
die "$@"   unless  ref($sub)
               &&  UNIVERSAL::isa($sub,"CODE");
while(<>) {
    if(  &$match()  ) {
        print "line $.: problem: $1\n$_\n";
    }
}


# Second way:
while(<>) {
    my $up= upcase $_;
    foreach my $p (  @problem  ) {
        if(  0 <= index($up,$p)  ) {
            print "line $.: problem: $p\n$_\n";
            last;
        }
    }
}
[download]

Both of my solutions differ from the original in that they only report one problem per line. My "second way" can be made like the original by simply removing the "last;" line. I see no simple way to make my "first way" like the original.

I apologize, but I didn't feel up to creating a test input file so I didn't test nor benchmark. I'd be interested in seeing test and benchmark results on real data.

Gosh dang, but you are right.

I just finished creating my sophisticated REs, saw your response, tried both of your versions. Your first one crashed, but your second ran just fine significantly faster than mine. Mine outran the original by a good factor as well. (I used the example code as a logfile to test. :-)

I guess the constant string search really is a nice win. Thanks for that lesson! :-)

I don't doubt that index() is faster than regexes; but it seems that, for regexes, the coderef approach would be slower than a qr// method:

@problem = map [$_, qr/\Q$_/i], @problem;

while (<>) {
    for my $p (@problem) {
        print "line: $. problem: $p->[0]\n" if /$p->[1]/;
    }
}
[download]

But I'm also too lazy to generate a good input file for benchmarking.

For an input problem, you could just use the input script again. That is what I did. Of course then you are really testing the time to compile code...

In any case there is overhead to entering a block, and to a hash lookup. Therefore I believe that tye's approach really is faster than your loop. But I went and tested it and found that between the three approaches the difference was all in the compilation. So I went for a longer file (a bit over 4 MB) and found that I was a bit under 27 seconds, tye just over 35, and yours 2 min, 22 seconds. I think tye's is faster than yours, and mine was not so bad after all. :-)

That said, perhaps I am not so ashamed to show the sophisticated method of building the RE, that coerces the RE engine into an optimization it should know about but does not (yet):

use strict;

my $match = &ret_match_any(
  "0  OBS",
  "AT LEAST",
  "EXTRANEOUS",
  "CARTESIAN",
  "CLOSING",
  "CONVERT",
  "DIVISION BY ZERO",
  "DOES NOT EXIST",
  "DUE TO LOOPING",
  "END OF MACRO",
  "ENDING EXECUTION",
  "ERROR",
  "ERRORABEND",
  "ERRORCHECK=STRICT",
  "EXCEED",
  "HANGING",
  "HAS 0 OBSERVATIONS",
  "ILLEGAL",
  "INCOMPLETE",
  "INVALID",
  "LOST CARD",
  "MATHEMAT",
  "MERGE STATEMENT",
  "MISSING",
  "MULTIPLE",
  "NOT FOUND",
  "NOT RESOLVED",
  "OBS=0",
  "REFERENCE",
  "REPEAT",
  "SAS CAMPUS DRIVE",
  "SAS SET OPTION OBS=0",
  "SAS WENT",
  "SHIFTED",
  "STOP",
  "TOO SMALL",
  "UNBALANCED",
  "UNCLOSED",
  "UNREF",
  "UNRESOLVED",
  "WARNING"
);

while(<>) {                        
  if ($_ =~ $match) {
    print "line $., file $ARGV, problem $1\n$_\n";
  }
}


# Takes a list of strings and returns an RE that matches any.
sub ret_match_any {
  my $match_str = &trie_strs(map quotemeta, @_);
  return qr /($match_str)/;
}

# Takes a list of escaped strings and returns a single string
# suitable for building a match in an efficient way.
# Works recursively by grouping strings that share one character
sub trie_strs {
  unless (@_) {
    return ();
  }
  my %rest;
  foreach my $str (@_) {
    if (length($str)) {
      my $chr = substr($str, 0, 1);
      if ("\\" eq $chr) {
        $chr = substr($str, 0, 2);
        push @{$rest{$chr}}, substr($str, 2);
      }
      else {
        push @{$rest{$chr}}, substr($str, 1);
      }
    }
    else {
      $rest{''} = [''];
    }
  }
  my @to_join;
  foreach my $chr (keys %rest) {
    my $list_ref = $rest{$chr};
    if (1 < @$list_ref) {
      push @to_join, $chr . &trie_strs(@$list_ref);
    }
    else {
      push @to_join, $chr . $list_ref->[0];
    }
  }
  if (1 < @to_join) {
    return '(?:' . (join '|', @to_join) . ')';
  }
  else {
    return $to_join[0];
  }     
}
[download]