in reply to Not sure what I am doing wrong

Contrary to what my fellow Monks are telling, it is not always a good idea to join your single regex-patterns with |and then use this combined regex on all your tests. Alternations in a regex are processed in O(n) time so increasing the number of alternations has a linear effect on your processing time.

To start with if you do 

my $RE = join '|', <RECORDS>;

[download]
and then later 
/$RE/;

[download]
for every test, Perl will have to re-compile your pattern every time. It is much better to pre-compile the pattern, by doing 
my $RE = join '|', <RECORDS>;
my $compiled_RE = qr/$RE/;

[download]
and then later 
/$compiled_RE/;

[download]
In that case you only pay the compilation cost once.

Even better is it to use a module such as Regexp::Assemble which can automatically compile for you a more efficient regex. Cutting down on the number of alternations will lead to a faster regex. I tried this in a program of mine and by using Regexp::Assemble managed to get a 30% speed increase when using real world data.

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

Replies are listed 'Best First'.
Re^2: Not sure what I am doing wrong
by massa (Hermit) on Nov 25, 2008 at 10:44 UTC
    As his original post had stated that he wanted to look for certain strings, not patterns, I think that the 
    chomp(my @RE = <RECORDS>);
my $RE = join '|', map quotemeta, @RE; $RE = qr/($RE)/;

    [download]
    will possibly be the most efficient matcher (at least on 5.10 ...), no?
    []s, HTH, Massa (κς,πμ,πλ)
[reply]
[d/l]
      Even then, the less alternations you have the faster your regex will run. Of course if the strings or patterns have nothing in common it makes no difference.

      I did a quick test and got the following results: 

                       Rate       Raw Regex Assembled Regex
Raw Regex       614/s              --             -9%
Assembled Regex 676/s             10%              --

      [download]
      The text to check was part of a play by Shakespeare, loaded into an array (1 line per element) and the regex used was 
      qr /this|that|here|there|where/;

      [download]
      Assembled into a more efficient regex it looks like: 
      qr /(?:th(?:ere|at|is)|w?here)/

      [download]
      Even by eliminating only ONE of the alternations (5 vs 4) a 10% speed-up was obtained.

      PS: I do not know about 5.10, this was tested in 5.8.

      CountZero

      A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

[reply]
[d/l]
[select]
[reply]

In Section Seekers of Perl Wisdom