Regex /o modifier: what bugs?

kcott has asked for the wisdom of the Perl Monks concerning the following question:

G'day All,

I've just read in "perlre: Other Modifiers":

o  - pretend to optimize your code, but actually introduce bugs
[download]

It doesn't say anything further about these "bugs". There is a link to "s/PATTERN/REPLACEMENT/msixpodualngcer in perlop" which I thought might have some more information. Alas, no; all it says about /o is:

"If you want the pattern compiled only once the first time the variable is interpolated, use the /o option."

I did have a more extensive look around this area of perlop: no /o bug info found.

So, if anyone can tell me about these bugs, that would be appreciated.

I am aware that when a regex is compiled with /o, it is fixed; that is, the regex /$var/o won't change even if $var does. That's been the case since (at least) Perl3 — I wonder if that's the "bugs" to which the doco refers.

Not really part of the question, more background really, but my "/o bug" query came about as I was doing some benchmarking (born of idle curiosity). Code and results in the spoiler for those interested.

#!/usr/bin/env perl

use strict;
use warnings;

use Benchmark 'cmpthese';

my $str = 'xyz';
my $re = qr{^xyz$};

cmpthese 0 => {
    use_m => \&use_m,
    as_qr => \&as_qr,
    use_o => \&use_o,
    raw_m => \&raw_m,
    raw_o => \&raw_o,
};

sub use_m {
    $str =~ m/$re/ and return;
}

sub as_qr {
    $str =~ $re and return;
}

sub use_o {
    $str =~ m/$re/o and return;
}

sub raw_m {
    $str =~ m/^xyz$/ and return;
}

sub raw_o {
    $str =~ m/^xyz$/o and return;
}
[download]

As usual, I ran the benchmark several times. Here's a fairly representative result:

            Rate as_qr use_m use_o raw_m raw_o
as_qr  2729934/s    --   -0%  -71%  -73%  -73%
use_m  2734042/s    0%    --  -71%  -73%  -73%
use_o  9454792/s  246%  246%    --   -6%   -7%
raw_m 10094766/s  270%  269%    7%    --   -1%
raw_o 10207253/s  274%  273%    8%    1%    --
[download]

Originally, I was looking at whether $re would be faster than m/$re/, given that the former required no interpolation; however, that was not the case, and both returned results that were not significantly different. I'm using Perl v5.36 which could be optimising this.

That was when I thought to try the /o modifier. I hadn't used this in years so I looked it up, which led to the bug question.

This could obviously do with more work: longer and more complex strings and regexes. Nonethless, an interesting result so far. I'll await news on the "bugs" before delving deeper.

— Ken

Comment on Regex /o modifier: what bugs? Select or Download Code

Replies are listed 'Best First'.
Re: Regex /o modifier: what bugs? by ikegami (Patriarch) on Dec 15, 2022 at 20:41 UTC
To my knowledge, `o` does what it says it does. I believe the author was hyperbolically stating that using `o` results in surprises, hard to test code, etc. This doesn't really belong in the doc, if so.	[reply] [d/l] [select]
Re^2: Regex /o modifier: what bugs? by Marshall (Canon) on Dec 16, 2022 at 05:43 UTC
For a regex intensive project many moons ago, I was running the same regex in loop against a big array of entries. I thought that this was the ideal place for the /o option, but I found that it made no performance difference at all. I remember reading somewhere that in some situations, Perl is smart enough to do /o automatically without being asked to do so. That appeared to be the case in my test. Its been so long ago, that I don't remember the exact regex being tested - could be that there wasn't a `$var` in it to begin with - don't remember.	[reply] [d/l]
Re^3: Regex /o modifier: what bugs? by ikegami (Patriarch) on Dec 21, 2022 at 22:20 UTC
Patterns that don't interpolate are always compiled at compile time. /o is useless for these. `$ perl -Mre=debug -c -e'/a/' Compiling REx "a" Final program: 1: EXACT <a> (3) 3: END (0) anchored "a" at 0..0 (checking anchored isall) minlen 1 -e syntax OK Freeing REx: "a"` [download] Each instance of a regex operator (m//, s///, qr//) that interpolates caches the last pattern it compiled in string and compiled form. If that instance is evaluated again, and if the generated pattern is the same as the previous one, recompilation is skipped. $ perl -Mre=debug -e'$x = "a"; for my $y (qw( b b a a )) { /$x/; /$y/; + }' 2>&1 \| grep -i comp Compiling REx "a" Compiling REx "b" Compiling REx "a" Skipping recompilation of unchanged REx "a" /$x/ same as last Compiling REx "b" Skipping recompilation of unchanged REx "b" /$y/ same as last Compiling REx "a" Skipping recompilation of unchanged REx "a" /$x/ same as last Compiling REx "a" Compiling REx "a" Skipping recompilation of unchanged REx "a" /$x/ same as last Compiling REx "a" Skipping recompilation of unchanged REx "a" /$y/ same as last [download] It still needs to perform the interpolation, and it needs to perform a string comparison, so it won't be as exactly as fast as using /o. But it should be close.	[reply] [d/l] [select]
Re^3: Regex /o modifier: what bugs? by LanX (Saint) on Dec 16, 2022 at 07:35 UTC
> I remember reading somewhere that in some situations, Perl is smart enough to do /o automatically without being asked to do so. I remember reading that this depends on the Perl version. And I think I read it here. At some point - long ago - Perl became capable to notice that the input pattern didn't change and reused the last compilation, IIRC. We should have extensive discussions on this in the archives. Cheers Rolf _{(addicted to the 𐍀𐌴𐍂𐌻 Programming Language :) Wikisyntax for the Monastery}	[reply]
Re^4: Regex /o modifier: what bugs? by choroba (Cardinal) on Dec 22, 2022 at 12:58 UTC
Re^2: Regex /o modifier: what bugs? by kcott (Archbishop) on Dec 15, 2022 at 20:49 UTC
G'day ikegami, ++ Thanks for the very quick response. That's good to know. — Ken	[reply]
Re: Regex /o modifier: what bugs? by Anonymous Monk on Dec 16, 2022 at 10:53 UTC
Based on the git history, someone just wanted to doc the other existing flags and their comment seems to be tongue in cheek, in that perl is already smart enough to do what it does, and you will likely just make your life harder forgetting that its on. As it says "The bottom line is that using /o is almost never a good idea."	[reply]