Regular expression operators (?{code}), (?!pattern), and (?...)

jffry has asked for the wisdom of the Perl Monks concerning the following question:

In another thread of mine someone responded with a snippet that I've been trying to wrap my head around ever since. This subject is a major tangent from the other thread, so I think it should be on its own. I hope that's OK.

Here is the original snippet. It is the contents of the s// regexp operator that has me completely stumped.

%d = (
    foo => sub { "[@_]"; },
    bar => sub { "(@_)"; },
);

$_ = 'foo fooz bar baz1';
s/(\w+)(?(?{ !$d{$1} })(?!))/$d{$1}->($1)/ge;
print "$_\n";
[download]

Here is the output.

[foo] [foo]z (bar) baz1
[download]

I added comments to the code in my attempt to understand what was happening. Most of my explanatory text is contained therein.

s/
    (\w+)   # One or more word chars in capture buffer 1. Got it.
    (   # Embedding a regexp modification character or what?
        # So everything within this set of parenthesis is
        # supposed to evaluate to either m, s, i, x, p, g, or c?
        # I don't get it.
        # Also, why is all this on the left side of s\\? Why are we
        # _matching_ this?
        ?(?{ !$d{$1} }) # This is really just used as a boolean to
                        # indicate that a hash key exists. I get it,
                        # kind of.
        (?!)    # From perldoc, "a zero-width negative look-ahead
                # assertion. I might understand this if only there was
                # a pattern after the "?!".
    )
/$d{$1}->($1)/gex;
print "$_\n";

# Here is my walkthrough for 'foo'.
# 1. (\w+) matches 'foo'.
# 2. $d{$1} is _true_. !_true_ = false.  In this case, false is "1".
# 3. (?!) says to *not* match a "1" followed by null. Maybe?
# 4. (?...) means use the 1 not followed by null as a regexp modifier
#    to 'foo'? What?!
[download]

A few of my big failures to comprehend I think are:

My thinking that the outer set of *(?...)* is the use of a regexp mod. But if it's not that, then what is it?
The *(?!)* without a pattern following it. Does he really want to match any '1' without a following null?
Why are these on the left side of the s//? Why are we matching something other than the word?

Thanks.

EDIT: I updated the $_ var to include the "baz1" string.

Comment on Regular expression operators (?{code}), (?!pattern), and (?...) Select or Download Code

Replies are listed 'Best First'.
Re: Regular expression operators (?{code}), (?!pattern), and (?...) by Khen1950fx (Canon) on Oct 31, 2009 at 01:07 UTC
When I get stuck on a regex, I use YAPE::Regex::Explain: `#!/usr/bin/perl use strict; use warnings; use ExtUtils::MakeMaker qw(prompt); use YAPE::Regex::Explain; my $re = prompt("Enter regexp: "); my $exp = YAPE::Regex::Explain->new($re)->explain; print $exp, "\n";` [download]	[reply] [d/l]
Re: Regular expression operators (?{code}), (?!pattern), and (?...) by ikegami (Patriarch) on Oct 31, 2009 at 02:06 UTC
The zero length pattern always matches, so `(?!)` never matches. 5.10 introduced alternative `(FAIL)`. `(?!)` is optimized into the same op as `(FAIL)` since the latter was introduced. `(?(?{ cond-expr })then-pat\|else-pat)` [download] is the conditional operator for regex. It can also appear in the form `(?(?{ cond-expr })then-pat)` [download] All of these are equivalent: `/ (?(?{ $d{$1} }) \| (FAIL) )/x` `/ (?(?{ !$d{$1} }) (FAIL) \| )/x` `/ (?(?{ !$d{$1} }) (FAIL) )/x` `/ (?(?{ !$d{$1} }) (?!) )/x` Update: Fixed to replace non-existent `(PASS)` with empty pattern.	[reply] [d/l] [select]
Re: Regular expression operators (?{code}), (?!pattern), and (?...) by AnomalousMonk (Archbishop) on Oct 31, 2009 at 01:49 UTC
Consider the output of your original regex and these two variations (I don't know where the `'baz1'` comes from in your output): >perl -wMstrict -le "my %d = ( foo => sub { \"[@_]\"; }, bar => sub { \"(@_)\"; }, ); $_ = 'foo fooz bar'; s/ (\w+) (?(?{ !$d{$1} }) (?!)) / $d{$1}->($1) /xge; print; " [foo] [foo]z (bar) >perl -wMstrict -le "my %d = ( foo => sub { \"[@_]\"; }, bar => sub { \"(@_)\"; }, ); $_ = 'foo fooz bar'; s/ (\w+) / $d{$1}->($1) /xge; print; " Use of uninitialized value in subroutine entry at ... Can't use string ("") as a subroutine ref while "strict refs" ... >perl -wMstrict -le "my %d = ( foo => sub { \"[@_]\"; }, bar => sub { \"(@_)\"; }, ); $_ = 'foo fooz bar'; s/ (\w+) / $d{$1} ? $d{$1}->($1) : $1 /xge; print; " [foo] fooz (bar) [download] Then consider the `(?(condition)yes-pattern)` construct in the Extended Patterns section of perlre and the fact that, if the null regex is always true, wouldn't the negative look-ahead to the null regex `(?!)` always be false? (Coupled with realizing what the logical value of `!$d{$1}` is when `$1` doesn't exist in the hash.)	[reply] [d/l] [select]