Re: Strange behaviour of the regex engine (pathological protection)

Perl optimizes certain nested loop regex constructs similar to (...*)* in order to avoid what is pathological behavior with most regex engines. For non-pathological cases, there is a small cost to this protection.

The reason the performance significantly decreases is because that crosses the point that triggers the warning "Complex regular subexpression recursion limit (32766) exceeded" (repeatedly).

- tye

Comment on Re: Strange behaviour of the regex engine (pathological protection)

Replies are listed 'Best First'.
Re^2: Strange behaviour of the regex engine (pathological protection) by thof (Initiate) on Jan 06, 2015 at 18:53 UTC
Many thanks! I didn't know about warnings. But I noticed new issue. The problem is that when this warning appears the regex doesn't match... I changed line to `print "not matched" if $TestString !~ m/^(a+\|b)$/;` and for string 256000 and above regex is not able to match. Also when I use possessive quantifiers (m/^(a++\|b)+$/) performance improves a lot but still it does not match for strings larger than 256000. I don't understand regex engine in Perl... I also checked the same regexes for Python and PHP (PCRE) and they work similar to Java. I doubt that this is bug in Perl and I cannot match big input strings. There must be something I don't know.	[reply] [d/l]
Re^3: Strange behaviour of the regex engine (bug) by tye (Sage) on Jan 06, 2015 at 21:46 UTC
If I were you, I would file this (the failure case) as a bug against Perl. If nothing else, it might lead to p5p giving you better information than I have off the top of my head. Best case, this 32k limit might be removed (a limit that seemed more reasonable when it was implemented but seems increasingly small as time passes). - tye	[reply]
Re^3: Strange behaviour of the regex engine (pathological protection) by Anonymous Monk on Jan 06, 2015 at 21:52 UTC
Many thanks! I didn't know about warnings. But I noticed new issue. The problem is that when this warning appears the regex doesn't match... It's not a new issue it's the same issue :) The recursion depth is limited to prevent catastrophic backtracking (mkay I think it's not actually recursion but it's called that in the docs). Doesn't Friedl describe that in his book? Add `use diagnostics;` to your program.	[reply] [d/l]