This is probably my least favorite piece of magic in the regular expression engine, but there you have it. If ever you seek to write a custom parse engine it is important to check whether you need to write:Repeated patterns matching zero-length substring WARNING: Difficult material (and prose) ahead. This section needs a rewrite. Regular expressions provide a terse and powerful programming language. As with most other power tools, power comes together with the ability to wreak havoc. A common abuse of this power stems from the ability to make infinite loops using regular expressions, with something as innocuous as: 'foo' =~ m{ ( o? )* }x; The o? can match at the beginning of 'foo', and since the position in the string is not moved by the match, o? would match again and again due to the * modifier. Another common way to create a similar cycle is with the looping modifier //g: @matches = ( 'foo' =~ m{ o? }xg ); or print "match: <$&>\n" while 'foo' =~ m{ o? }xg; or the loop implied by split(). However, long experience has shown that many programming tasks may be significantly simplified by using repeated subexpressions which may match zero-length substrings, with a simple example being: @chars = split //, $string; # // is not magic in +split ($whitewashed = $string) =~ s/()/ /g; # parens avoid magic +s// / Thus Perl allows the /()/ construct, which forcefully breaks the infinite loop. The rules for this are different for lower-level loops given by the greedy modifiers *+{}, and for higher-level ones like the /g modifier or split() operator. The lower-level loops are interrupted when it is detected that a repeated expression did match a zero-length substring, thus m{ (?: NON_ZERO_LENGTH | ZERO_LENGTH )* }x; is made equivalent to m{ (?: NON_ZERO_LENGTH )* | (?: ZERO_LENGTH )? }x; The higher level-loops preserve an additional state between iterations: whether the last match was zero- length. To break the loop, the following match after a zero-length match is prohibited to have a length of zero. This prohibition interacts with backtracking (see the section on Backtracking), and so the second best match is chosen if the best match is of zero length. Say, $_ = 'bar'; s/\w??/<$&>/g; results in "<<b><><a><><r><>">. At each position of the string the best match given by non-greedy ?? is the zero- length match, and the second best match is what is matched by \w. Thus zero-length matches alternate with one- character-long matches. Similarly, for repeated m/()/g the second-best match is the match at the position one notch further in the string. The additional state of being matched with zero-length is associated to the matched string, and is reset by each assignment to pos().
from time to time.pos($string) = pos($string); # Resets an internal flag
In reply to Re (tilly) 3: Bother (Re: Re: Split and empty strings)
by tilly
in thread Split and empty strings
by andye
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |