Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have an installation script which behaviour differs undesired depending of prior execution of unrelated regex.

The script restarts services after completing the installation, but when

$supported =~ /\Q4.1.2/

is not executed not all services are started.

I have absolutely no clue as to why not executing an otherwise unrelated regex changes the behaviour of regexs inside a function called later.

Is this behaviour caused by using an undefined variable in a regex?

It looks mighty strange and I have worked around the issue by not using an undefined variable in a regex

I have reduced the installation script to

use v5.30.0; my @stoppedGeneralServices = ( 'OP Mover', 'OP Monitor'); my @stoppedStandardServices = ( 'OP OPC Client', 'OP Calculation Serve +r', 'OP Data Server', 'OP Configuration Server', 'OP Log Server', 'OP + Time Server', 'ONC RPC PortMapper'); my $supported = '4.0.0,4.0.1,4.1.0,4.1.1,4.1.2'; $supported =~ /\Q4.1.2/ if @ARGV; say "Start Basic Standard Services:"; startStandardServices( 'Config'); say "Start Remaining Standard Services:"; startStandardServices(); say "Start General Services:"; startGeneralServices(); say "Installation completed"; sub startStandardServices { my $last = shift; my $name; while (@stoppedStandardServices) { $name = pop @stoppedStandardServices; say ' Starting $name=\'', $name, '\' $last=', defined $last ? + "'$last'" : 'undef', ' ($name=~/$last/i)=', ($name =~ /$last/i) ? 1 +: 0; last if $name =~ /$last/i; } } sub startGeneralServices { my $last = shift; my $name; while (@stoppedGeneralServices) { $name = pop @stoppedGeneralServices; say ' Starting $name=\'', $name, '\' $last=', defined $last ? + "'$last'" : 'undef', ' ($name=~/$last/i)=', ($name =~ /$last/i) ? 1 +: 0; last if $name =~ /$last/i; } }

When executed without arguments the output is:

D:\>surprise.pl Start Basic Standard Services: Starting $name='ONC RPC PortMapper' $last='Config' ($name=~/$last/i) +=0 Starting $name='OP Time Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Log Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Configuration Server' $last='Config' ($name=~/$la +st/i)=1 Start Remaining Standard Services: Starting $name='OP Data Server' $last=undef ($name=~/$last/i)=1 Start General Services: Starting $name='OP Monitor' $last=undef ($name=~/$last/i)=1 Installation completed

When executed with an argument the output is:

D:\>surprise.pl all Start Basic Standard Services: Starting $name='ONC RPC PortMapper' $last='Config' ($name=~/$last/i) +=0 Starting $name='OP Time Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Log Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Configuration Server' $last='Config' ($name=~/$la +st/i)=1 Start Remaining Standard Services: Starting $name='OP Data Server' $last=undef ($name=~/$last/i)=0 Starting $name='OP Calculation Server' $last=undef ($name=~/$last/i) +=0 Starting $name='OP OPC Client' $last=undef ($name=~/$last/i)=0 Start General Services: Starting $name='OP Monitor' $last=undef ($name=~/$last/i)=0 Starting $name='OP Mover' $last=undef ($name=~/$last/i)=0 Installation completed

Replies are listed 'Best First'.
Re: Side effect of using undefined variable in regex (updated)
by haukex (Archbishop) on Jul 01, 2019 at 08:15 UTC
    Is this behaviour caused by using an undefined variable in a regex?

    Yes, more specifically a variable that evaluates to the empty string. From Regexp Quote Like Operators:

    If the PATTERN evaluates to the empty string, the last successfully matched regular expression is used instead. ... If no match has previously succeeded, this will (silently) act instead as a genuine empty pattern (which will always match).

    Update: Added the second half of the perlop quote, and see also my reply further down in this thread.

      When I realized that, I thought this explained the behaviour. But the problem is, the regex at the top of the script is not the last regex, since there has been a successful match with /$last/ previously. I have tried replacing the "useless" match with this:

      my $supported = '4.0.0,4.0.1,4.1.0,4.1.1,4.1.2'; $supported =~ /\Q4.1.2\E(?{say '>>> In the regex'})/ if @ARGV; "4.1.2" =~ //;
      And the message does appear twice at the start, but does not appear when the special behaviour of m// is triggered.

      In all cases, use warnings; would have prevented this issue though.

      Edit: ... the (?{ }) block is never reached in the call to /$last/ because the first half of the regex does not match. PEBKAC

        Ok, I've looked at it a little further now. I omitted a bit in my quote from perlop:

        If the PATTERN evaluates to the empty string, the last successfully matched regular expression is used instead. ... If no match has previously succeeded, this will (silently) act instead as a genuine empty pattern (which will always match).

        I think there is something omitted from the documentation - AnomalousMonk describes this as well:

        $ perl -le'print"x"=~//?"yes":"no"; "y"=~/y/; print"x"=~//?"yes":"no"' yes no $ perl -le'print"x"=~//?"yes":"no";{"y"=~/y/} print"x"=~//?"yes":"no"' yes yes

        So in other words, the "last successfully matched regular expression" means the "last successfully matched regular expression in this scope", which explains why the successful match when $last is "Config" doesn't affect the other matches. I stumbled on this at first; I think it might be worth a documentation patch...

        Anyway, that means in the OP's example, we can remove startStandardServices('Config') since that's working as expected, and since the other two calls of startStandardServices and startGeneralServices are seeing the same issue each, the script can be reduced to:

        use warnings; use 5.014; my @stoppedGeneralServices = ('OP Mover', 'OP Monitor'); my $supported = '4.0.0,4.0.1,4.1.0,4.1.1,4.1.2'; $supported =~ /\Q4.1.2/ if @ARGV; startGeneralServices(); sub startGeneralServices { my $last = shift; my $name; while (@stoppedGeneralServices) { $name = pop @stoppedGeneralServices; say ' Starting $name=\'', $name, '\' $last=', defined $last ? "'$last'" : 'undef', ' ($name=~/$last/i)=', ($name =~ /$last/i) ? 1 : 0; last if $name =~ /$last/i; } }

        And taking that a few steps further:

        use warnings; use 5.014; '4.1.1,4.1.2' =~ /\Q4.1.2/ if @ARGV; my $last = ''; my $name = 'OP Mover'; say '$name=\'', $name, '\' $last=', defined $last ? "'$last'" : 'undef', ', $name=~/$last/i = ', ($name =~ /$last/i) ? 1 : 0;
        $ perl 11102215.pl $name='OP Mover' $last='', $name=~/$last/i = 1 $ perl 11102215.pl all $name='OP Mover' $last='', $name=~/$last/i = 0

        Which shows that:

        • If @ARGV is false, there is no previously executed regex, so /$last/ aka // acts like the empty pattern, which always matches.
        • If @ARGV is true, then // means to use the previous successful regex, /\Q4.1.2/, which does not match the string 'OP Mover'.

      Thanks

      This behaviour corresponds exactly to what I experience in my further reduced script:

      use v5.30.0; 'ABC' =~ /B/ if @ARGV; my $last; say ' (\'NAME\'=~/$last/i)=', 'NAME' =~ /$last/i ? 1 : 0;

      Output when called without arguments:

      D:\>surprise.pl ('NAME'=~/$last/i)=1

      Output when called with arguments:

      D:\>surprise.pl all ('NAME'=~/$last/i)=0
Re: Side effect of using undefined variable in regex
by AnomalousMonk (Archbishop) on Jul 01, 2019 at 11:01 UTC

    I think haukex is right. I think a narrative of what's happening might be something like this:

    c:\@Work\Perl\monks>perl -Mstrict -le "my @services = qw(BANG one two three STOP four five six); ;; my $side_effect = 'xxx'; $side_effect =~ /xxx/ if @ARGV; ;; func('STOP'); ;; func(); ;; sub func { my $done = shift; while (my $serv = pop @services) { printf qq{$serv }; last if $serv =~ /$done/i } print ''; } " six five four STOP three
    In this example (no side-effect match at all), the first call to  func('STOP') loops until it sees a match on STOP, as expected. The second call to  func() (warnings not enabled!) matches on an empty pattern ( /undef/ -> //) because there has been no previous successful match in the dynamic scope of the matching operator. Matching on // always matches, so the while-loop terminates after three.

    c:\@Work\Perl\monks>perl -Mstrict -le "my @services = qw(BANG one two three STOP four five six); ;; my $side_effect = 'xxx'; $side_effect =~ /xxx/ if @ARGV; ;; func('STOP'); ;; func(); ;; sub func { my $done = shift; while (my $serv = pop @services) { printf qq{$serv }; last if $serv =~ /$done/i } print ''; } " ARGUMENT six five four STOP three two one BANG
    In this example, identical except that an argument is passed to the script and a successful side-effect match occurs, the first call to  func('STOP') runs as before. The second call to  func() matches on the pattern  /xxx/ because there has been a previous successful match in the dynamic scope of the matching operator, the side-effect match /xxx/. Matching on /xxx/ never succeeds, so the while-loop runs to completion: BANG.

    The critical point is that the match in  func() during the second call is not in the dynamic scope of the match during the first call, but only in the dynamic scope of the side-effect match, should it have occurred.

    If that makes any sense...

    Run under Perl version 5.8.9 without warnings.

    Update: A few trivial wording changes (update: and a spelling correction).


    Give a man a fish:  <%-{-{-{-<

Re: Side effect of using undefined variable in regex
by Marshall (Canon) on Jul 02, 2019 at 01:15 UTC
    When I ran your code under Perl 5.10 with warnings enabled, I got:
    C:\PerlProjects\Monks>perl regextest.pl Start Basic Standard Services: Starting $name='ONC RPC PortMapper' $last='Config' ($name=~/$last/i) +=0 Starting $name='OP Time Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Log Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Configuration Server' $last='Config' ($name=~/$la +st/i)=1 Start Remaining Standard Services: Use of uninitialized value $last in regexp compilation at regextest.pl + line 27. Starting $name='OP Data Server' $last=undef ($name=~/$last/i)=1 Use of uninitialized value $last in regexp compilation at regextest.pl + line 28. Start General Services: Use of uninitialized value $last in regexp compilation at regextest.pl + line 38. Starting $name='OP Monitor' $last=undef ($name=~/$last/i)=1 Use of uninitialized value $last in regexp compilation at regextest.pl + line 39. Installation completed C:\PerlProjects\Monks>perl regextest.pl asdf asdf Start Basic Standard Services: Starting $name='ONC RPC PortMapper' $last='Config' ($name=~/$last/i) +=0 Starting $name='OP Time Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Log Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Configuration Server' $last='Config' ($name=~/$la +st/i)=1 Start Remaining Standard Services: Use of uninitialized value $last in regexp compilation at regextest.pl + line 27. Starting $name='OP Data Server' $last=undef ($name=~/$last/i)=0 Use of uninitialized value $last in regexp compilation at regextest.pl + line 28. Use of uninitialized value $last in regexp compilation at regextest.pl + line 27. Starting $name='OP Calculation Server' $last=undef ($name=~/$last/i) +=0 Use of uninitialized value $last in regexp compilation at regextest.pl + line 28. Use of uninitialized value $last in regexp compilation at regextest.pl + line 27. Starting $name='OP OPC Client' $last=undef ($name=~/$last/i)=0 Use of uninitialized value $last in regexp compilation at regextest.pl + line 28. Start General Services: Use of uninitialized value $last in regexp compilation at regextest.pl + line 38. Starting $name='OP Monitor' $last=undef ($name=~/$last/i)=0 Use of uninitialized value $last in regexp compilation at regextest.pl + line 39. Use of uninitialized value $last in regexp compilation at regextest.pl + line 38. Starting $name='OP Mover' $last=undef ($name=~/$last/i)=0 Use of uninitialized value $last in regexp compilation at regextest.pl + line 39. Installation completed
    I added:
    sub startStandardServices { my $last = shift; $last //='EVERYTHING'; #ADDED sub startGeneralServices { my $last = shift; $last //='EVERYTHING'; #ADDED
    And got the same results with or without command line args. I think the logic needs to be modified somewhat to get what you want. the main thing is get rid of this "undef in regex" warning.
    C:\PerlProjects\Monks>perl regextest.pl Start Basic Standard Services: Starting $name='ONC RPC PortMapper' $last='Config' ($name=~/$last/i) +=0 Starting $name='OP Time Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Log Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Configuration Server' $last='Config' ($name=~/$la +st/i)=1 Start Remaining Standard Services: Starting $name='OP Data Server' $last='EVERYTHING' ($name=~/$last/i) +=0 Starting $name='OP Calculation Server' $last='EVERYTHING' ($name=~/$ +last/i)=0 Starting $name='OP OPC Client' $last='EVERYTHING' ($name=~/$last/i)= +0 Start General Services: Starting $name='OP Monitor' $last='EVERYTHING' ($name=~/$last/i)=0 Starting $name='OP Mover' $last='EVERYTHING' ($name=~/$last/i)=0 Installation completed <c> C:\PerlProjects\Monks>perl regextest.pl asdf asdf Start Basic Standard Services: Starting $name='ONC RPC PortMapper' $last='Config' ($name=~/$last/i) +=0 Starting $name='OP Time Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Log Server' $last='Config' ($name=~/$last/i)=0 Starting $name='OP Configuration Server' $last='Config' ($name=~/$la +st/i)=1 Start Remaining Standard Services: Starting $name='OP Data Server' $last='EVERYTHING' ($name=~/$last/i) +=0 Starting $name='OP Calculation Server' $last='EVERYTHING' ($name=~/$ +last/i)=0 Starting $name='OP OPC Client' $last='EVERYTHING' ($name=~/$last/i)= +0 Start General Services: Starting $name='OP Monitor' $last='EVERYTHING' ($name=~/$last/i)=0 Starting $name='OP Mover' $last='EVERYTHING' ($name=~/$last/i)=0 Installation completed