brian_d_foy has asked for the wisdom of the Perl Monks concerning the following question:

I ran into a new question in our Learning Perl class today. A student tried to combine a previous example with the redo stuff we had just covered. I didn't see anything in Perlmonks or in Google that discussed this, but if it has please point it out to me. :)

While using a for loop, use s/// to replace a character with nothing, but remember the character. Redo the block if the character is an e. I expected this to make an infinite loop because redo would stay in the same dynamic scope, but I see that's not true in practice:

#!/usr/bin/perl use strict; use warnings; for( $_ = "fred"; s/(.)//; ) { print "I saw the character [$&]\n"; redo if $& eq 'e'; }

The output shows that it sees the e and does the redo, but when it goes through the block again, the value in $& has disappeared (which is why it isn't an infinite loop).

I saw the character [f] I saw the character [r] I saw the character [e] Use of uninitialized value in concatenation (.) or string at test line + 7. I saw the character [] Use of uninitialized value in string eq at test line 8. I saw the character [d]

Let's try it with the match operator. I see the same effect, but also notice that the value of $i is just fine.

#!/usr/bin/perl use strict; use warnings; for( my $i = 1; $i =~ m/\d/; $i++) { print "I saw [$i]\n"; redo if $& == 5; last if $i == 10; }

Output:

I saw [1] I saw [2] I saw [3] I saw [4] I saw [5] I saw [5] Use of uninitialized value in numeric eq (==) at test2 line 8. I saw [6] I saw [7] I saw [8] I saw [9] I saw [10]

perlvar says that $& is actually dynamically-scoped to the current block, but I'm also assuming that redo stays in the same scope, so I expect that shouldn't be a problem since I think that the current block should be the one that contains the for.

The string matched by the last successful pattern match (not counting any matches hidden within a BLOCK or eval() enclosed by the current BLOCK). (Mnemonic: like & in some editors.) This variable is read-only and dynamically scoped to the current BLOCK.

So, I suspected that redo was actually creating a new dynamic scope. I add another lexical variable to the test portion of the for.

#!/usr/bin/perl use strict; use warnings; for( my $i = 1; $i =~ m/\d/, my $j = $i**2; $i++) { print "I saw [$i] and [$j]\n"; redo if $& == 5; last if $i == 10; }

The output shows that redo hides the value of $j too:

I saw [1] and [1] I saw [2] and [4] I saw [3] and [9] I saw [4] and [16] I saw [5] and [25] Use of uninitialized value in concatenation (.) or string at test2 lin +e 7. I saw [5] and [] Use of uninitialized value in numeric eq (==) at test2 line 8.I saw [6 +] and [36] I saw [7] and [49] I saw [8] and [64] I saw [9] and [81] I saw [10] and [100]
Here it is without the for loop:
#!/usr/bin/perl use strict; use warnings; $_ = 123; while( m/\d/g ) { print "I saw [$&]\n"; redo if $& == 1; last; }

The output shows he same effect:

I saw [1] Use of uninitialized value in concatenation (.) or string at test3 lin +e 9. I saw []
So, is this the right behavior? If not, which part is not acting right? The lexical scope or redo?
--
brian d foy <brian@stonehenge.com>
Subscribe to The Perl Review

Replies are listed 'Best First'.
Re: Does redo create a new dynamic scope?
by ikegami (Patriarch) on Jul 18, 2007 at 21:01 UTC

    Some more info:

    I also came up with a concise, minimalistic problem definition:

    Why do these two loops have different outputs?

    { print("norm: "); local $_ = 'fred'; for (; /./g ; print("<$&>"),1) { } print("\n"); } { print("next: "); local $_ = 'fred'; for (; /./g ; print("<$&>"),1) { next } print("\n"); }
    norm: <f><r><e><d> next: <><><><>
      It seems to me that's because of
      if (cxix < cxstack_ix) dounwind(cxix);

      in PP(pp_next) and PP(pp_redo) in pp_ctl.c. Debugging reveals an additional BLOCK frame which is built around the scope in which a next or redo is found:

      qwurx [shmem] ~ > perl -D4 { print("norm: "); local $_ = 'fred'; for (; /./g ; print("<$&>\n"),1) { } print("\n"); } { print("next: "); local $_ = 'fred'; for (; /./g ; print("<$&>\n"),1) { next } print("\n"); } __END__ (some lines skipped) EXECUTING... (next.pl:0) ENTER scope 2 at pp_hot.c:1694 Entering block 0, type BLOCK (next.pl:2) ENTER scope 3 at pp_ctl.c:1798 (next.pl:2) ENTER scope 4 at pp_ctl.c:1800 Entering block 1, type LOOP (next.pl:4) ENTER scope 5 at pp_ctl.c:1798 (next.pl:4) ENTER scope 6 at pp_ctl.c:1800 Entering block 2, type LOOP norm: <f> <r> <e> <d> (next.pl:4) POPBLOCK scope 6 at pp_ctl.c:1817 Leaving block 2, type LOOP (next.pl:4) LEAVE scope 6 at pp_ctl.c:1843 (next.pl:4) LEAVE scope 5 at pp_ctl.c:1844 (next.pl:2) POPBLOCK scope 4 at pp_ctl.c:1817 Leaving block 1, type LOOP (next.pl:2) LEAVE scope 4 at pp_ctl.c:1843 (next.pl:2) LEAVE scope 3 at pp_ctl.c:1844 (next.pl:8) ENTER scope 3 at pp_ctl.c:1798 (next.pl:8) ENTER scope 4 at pp_ctl.c:1800 Entering block 1, type LOOP (next.pl:10) ENTER scope 5 at pp_ctl.c:1798 (next.pl:10) ENTER scope 6 at pp_ctl.c:1800 Entering block 2, type LOOP (next.pl:10) ENTER scope 7 at pp_hot.c:1694 Entering block 3, type BLOCK (next.pl:10) (Found loop #2) Unwinding block 3, type BLOCK (next.pl:10) TOPBLOCK scope 6 at pp_ctl.c:2073 next: <> (next.pl:10) ENTER scope 7 at pp_hot.c:1694 Entering block 3, type BLOCK (next.pl:10) (Found loop #2) Unwinding block 3, type BLOCK (next.pl:10) TOPBLOCK scope 6 at pp_ctl.c:2073 <> (next.pl:10) ENTER scope 7 at pp_hot.c:1694 Entering block 3, type BLOCK (next.pl:10) (Found loop #2) Unwinding block 3, type BLOCK (next.pl:10) TOPBLOCK scope 6 at pp_ctl.c:2073 <> (next.pl:10) ENTER scope 7 at pp_hot.c:1694 Entering block 3, type BLOCK (next.pl:10) (Found loop #2) Unwinding block 3, type BLOCK (next.pl:10) TOPBLOCK scope 6 at pp_ctl.c:2073 <> (next.pl:10) POPBLOCK scope 6 at pp_ctl.c:1817 Leaving block 2, type LOOP (next.pl:10) LEAVE scope 6 at pp_ctl.c:1843 (next.pl:10) LEAVE scope 5 at pp_ctl.c:1844 (next.pl:8) POPBLOCK scope 4 at pp_ctl.c:1817 Leaving block 1, type LOOP (next.pl:8) LEAVE scope 4 at pp_ctl.c:1843 (next.pl:8) LEAVE scope 3 at pp_ctl.c:1844 (next.pl:0) POPBLOCK scope 2 at pp_hot.c:1804 Leaving block 0, type BLOCK (next.pl:0) LEAVE scope 2 at pp_hot.c:1843 (next.pl:0) Setting up jumplevel bfdb9400, was 816619c (next.pl:0) LEAVE scope 1 at perl.c:645

      update: Things become more clear if we change the 'next' block as follows:

      { print("next: "); local $_ = 'fred'; for (; /./g ; print("<$&>\n"),1) { print "<$&>\n"; next; } print("\n"); }

      which yields

      EXECUTING... (next.pl:0) ENTER scope 2 at pp_hot.c:1694 Entering block 0, type BLOCK (next.pl:2) ENTER scope 3 at pp_ctl.c:1798 (next.pl:2) ENTER scope 4 at pp_ctl.c:1800 Entering block 1, type LOOP (next.pl:4) ENTER scope 5 at pp_ctl.c:1798 (next.pl:4) ENTER scope 6 at pp_ctl.c:1800 Entering block 2, type LOOP (next.pl:4) ENTER scope 7 at pp_hot.c:1694 Entering block 3, type BLOCK next: <f> (next.pl:6) (Found loop #2) Unwinding block 3, type BLOCK (next.pl:6) TOPBLOCK scope 6 at pp_ctl.c:2073 <> (next.pl:4) ENTER scope 7 at pp_hot.c:1694 Entering block 3, type BLOCK <r> (next.pl:6) (Found loop #2) Unwinding block 3, type BLOCK (next.pl:6) TOPBLOCK scope 6 at pp_ctl.c:2073 <> ...

      So, the print in the for() condition is in scope 6, while the print just before next is in scope 7...

      <update>

      What is happening in this loop is that the /./g is evaluated in scope 6. Then the inner loop block is executed (scope 7, block 3). Then that block is left, and all lexical (and match) variables are restored. After that the last part of the for(;;) conditional construct is executed - the print("<$&>\n"),1 - at which time $& happens to have been reset already, because the inner scope has been left, as can be seen with the following snippet:

      { print("next: "); local $_ = 'fred'; for ( print "begin\n"; /./g, print("1<$&>\n") and $& ? 1 : 0 ; print("2<$&>\n"),1) { print "3<$&>\n"; next; } print("\n"); } __END__ next: begin 1<f> 3<f> 2<> 1<r> 3<r> 2<> 1<e> 3<e> 2<> 1<d> 3<d> 2<> 1<>

      Run it under -D4 :-)

      </update>

      --shmem

      _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                    /\_¯/(q    /
      ----------------------------  \__(m.====·.(_("always off the crowd"))."·
      ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
Re: Does redo create a new dynamic scope? (my)
by tye (Sage) on Jul 18, 2007 at 19:17 UTC

    Sorry for the very short note (I'll try to expand on this when I have more time).

    I suspect that the new scope is required for the sake of any lexical variables in the block (and because re-initialization of lexicals happens at the end of the block).

    I'm interested in why the match variables are not inherited in this new scope since the match operation was performed from outside of the curly braces. It is my understanding that some control structures in Perl introduce an implicit scope beyond the one(s) designated by the braces, but I didn't think any of them altered the brace-designated lexical scope. And match variables should survive into nested scopes.

    - tye        

Re: Does redo create a new dynamic scope?
by neosamuri (Friar) on Jul 18, 2007 at 19:35 UTC

    The $& is not set on the redone block. Look at perlre

    The numbered match variables ($1, $2, $3, etc.) and the related punctuation set ($+ , $& , $` , $' , and $^N ) are all dynamically scoped until the end of the enclosing block or until the next successful match, whichever comes first.

    This block should give the behavior you were expecting, by storing the $& from the previous run.

    #!/usr/bin/perl use strict; use warnings; my $b; for( $_ = "fred"; s/(.)//; ) { $b = $& if $&; print "I saw the character [$b]\n"; redo if $b eq 'e'; }

      Probably so, but why was the enclosing block exited? Keep in mind the block enclosing the regexp is NOT the curlies, it's a larger one that includes the whole for loop.

      According to what you are saying, the output of the loops below should both be the same ([a][]). They should both be the same ([a][a]), but the second is producing the wrong ouput. The answer to "why do these two loops have different outputs" is answer to the OP.

      print("norm: "); for (my $i; $i ? $i<2 : 'a'=~/./g; ++$i) { print("[$&]"); } print("\n"); print("next: "); for (my $i; $i ? $i<2 : 'a'=~/./g; ++$i) { print("[$&]"); next; } print("\n");
      norm: [a][a] next: [a][]

      Update: As ikegami illustrates below, neosamurai and I were in agreement about the answer to the wrong question. It did seem a bit odd that someone as knowledgeable as brian_d_foy would be asking a question with an answer that seemed so clear to me...

      My original post remains below.


      neosamurai is right. $& goes out of scope when redo moves code execution back to the top of the block. redo doesn't reevaluate the loop's conditional (where we set $&), so a new value for $& is never set.

      There doesn't seem to be anything special about a for loop, as the same behavior can be seen in a while loop.

      for( $_ = 'fred'; print "loop\n" and s/(.)//; ) { print qq'I saw [$&]\n'; redo if $& eq 'e'; } __END__ loop I saw [f] loop I saw [r] loop I saw [e] I saw [] loop I saw [d] loop
      $_ = 'fred'; while( print "loop\n" and s/(.)// ) { print qq'I saw [$&]\n'; redo if $& eq 'e'; } __END__ loop I saw [f] loop I saw [r] loop I saw [e] I saw [] loop I saw [d] loop


      TGI says moo

      Yes, but redo bypasses the conditional. There is no match and execution stays in the block. You shouldn't *have* to save $&.

      Anno

Re: Does redo create a new dynamic scope? (yes)
by shmem (Chancellor) on Jul 19, 2007 at 08:17 UTC
    Saying next or redo skips the execution of - what? the current block? Right; but that block cannot be the outer block; that would be last. So these OPs need something to skip, and that's why an implicit block (and scope) is created inside a block in which next or redo are found.

    <update>

    In your example (label added)

    LOOP: for( my $i = 1; $i =~ m/\d/, my $j = $i**2; $i++) { print "I saw [$i] and [$j]\n"; redo LOOP if $& == 5; last if $i == 10; }

    the redo goes straight to the LOOP label, so the scope of the loop is re-entered, and my-variables get reset (match variables too), as expected. But the execution of the "loop control block" conditional is skipped, which is why $j is undef after the redo.

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
      But the execution of the "loop control block" conditional is skipped, which is why $j is undef after the redo.

      This definitely makes sense. I don't know about other readers of this thread but you saved my sanity with this one :)

        I don't know about my sanity, but this has certainly been one of the most interesting threads of late. It's interesting to see some of Perl elites digging under the hood to figure out some perplexing behavior. One of the reason I read this site is to learn tidbits like this.
      Score one for Perl Best Practices (pg 129):

      Label every loop that is exited explicitly, and use the label with every next, last, or redo.

      Ironic given that we were debating the appropriateness of the title "Perl Best Practices" a while back in this thread: Re: Returning undef: The point I would like Damian to reconsider.