Aighearach has asked for the wisdom of the Perl Monks concerning the following question:

i tried to find the answer by RTFM, but have you ever tried searching the documentation for "for"?!

Anyhow, here is my problem:

for LIST and for (;;) are behaving differently from what I would expect. Granted, I've never tried using for (;;) in Perl before, but I've used it a lot in C and the chess client I am working on seemed to call for it. Also, I've only used anonymous subrutines a couple times, and never as closures. After reading the section of perlfaq7 on closures, I think I understand it, but I don't understand what is going wrong in my code. So obviously I don't. Here is the code: 

my ( @ref_list, $code_ref, $i );

#pass one
#for SCALAR LIST
for $i (1..8) {
    push( @ref_list, sub { print "    \$i == $i\n" } );
}

print "for SCALAR LIST\n";
while ( $code_ref = pop @ref_list ) {
    &$code_ref;
}

#pass 2
#for INIT ; TEST ; INCREMENT
for ( $i = 1; $i <= 8; $i++ ) { 
    push( @ref_list, sub { print "    \$i == $i\n" } );
}

print "for (;;)\n";
while ( $code_ref = pop @ref_list ) {
    &$code_ref;
}

[download]

Pass 1 works as planned, but pass 2 contains a logic error. All the code refs in pass 2 output the same thing. If somebody could explain this, I would be very grateful. 

Paris Sinclair    |    4a75737420416e6f74686572
pariss@efn.org    |    205065726c204861636b6572
http://sinclairinternetwork.com

Replies are listed 'Best First'.
Closures (was Re: for loops)
by merlyn (Sage) on Aug 20, 2000 at 21:48 UTC
    One of the things when I get an unexpected closure binding is to print both the reference to the variables I'm expecting to be bound, and the resulting coderef. By watching the addresses change (or not) I can tell whether the variables have independant binding (or not). For example: 
    for ($i = 1; $i <= 8; $i++) {
  print "\\\$i is ", \$i, "\n";
  my $c = sub { print "\$i is $i\n" };
  print "\$c is $c\n";
  push @codes, $c;
}

    [download]
    For this one, I got: 
    \$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)
\$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)
\$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)
\$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)
\$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)
\$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)
\$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)
\$i is SCALAR(0x80ce98c)
$c is CODE(0x80cea04)

    [download]
    showing that we are neither looking at distinct $i nor a closure (the addresses are all identical). Now, let's add a my declaration: 
    for (my $i = 1; $i <= 8; $i++) {
  print "\\\$i is ", \$i, "\n";
  my $c = sub { print "\$i is $i\n" };
  print "\$c is $c\n";
  push @codes, $c;
}

    [download]
    and notice that we now get a closure (the coderef addresses change): 
    \$i is SCALAR(0x80ce974)
$c is CODE(0x80c83ac)
\$i is SCALAR(0x80ce974)
$c is CODE(0x80d22f8)
\$i is SCALAR(0x80ce974)
$c is CODE(0x80d2364)
\$i is SCALAR(0x80ce974)
$c is CODE(0x80d23d0)
\$i is SCALAR(0x80ce974)
$c is CODE(0x80d243c)
\$i is SCALAR(0x80ce974)
$c is CODE(0x80d24a8)
\$i is SCALAR(0x80ce974)
$c is CODE(0x80d2514)
\$i is SCALAR(0x80ce974)
$c is CODE(0x80d2580)

    [download]
    So, even though these are all closures, they are all linked to the same now changed $i. This will protect them for when the $i goes out of scope, but they still all share the same one. What we need is to have a "different" $i each time: 
     foreach my $i (1..8) {
  print "\\\$i is ", \$i, "\n";
  my $c = sub { print "\$i is $i\n" };
  print "\$c is $c\n";
  push @codes, $c;
}

    [download]
    Ahh, there we go, now we got it: 
    \$i is SCALAR(0x80c83ac)
$c is CODE(0x80ce95c)
\$i is SCALAR(0x80d21b8)
$c is CODE(0x80d21c4)
\$i is SCALAR(0x80d2230)
$c is CODE(0x80d223c)
\$i is SCALAR(0x80d22a8)
$c is CODE(0x80d22b4)
\$i is SCALAR(0x80d2320)
$c is CODE(0x80d232c)
\$i is SCALAR(0x80d2398)
$c is CODE(0x80d23a4)
\$i is SCALAR(0x80d2410)
$c is CODE(0x80d241c)
\$i is SCALAR(0x80d2488)
$c is CODE(0x80d2494)

    [download]
    See the distinct $i variables, as well as the closures created for them!

    Hopefully, this kind of reference-to examination will give you more clues faster when you can't see why things are shared or not. Very helpful for those $x won't stay shared diagnostics.

    -- Randal L. Schwartz, Perl hacker

[reply]
[d/l]
[select]
      Thanks, Randal. Earlier today I was using that technique to flush keys and values out of a poorly documented object (Gtk), but of course it didn't occur to me to use it here. "Very helpful" indeed, hopefully it will help make my debugging toolkit more genericized. 
      Paris Sinclair    |    4a75737420416e6f74686572
pariss@efn.org    |    205065726c204861636b6572
http://sinclairinternetwork.com
[reply]
(Ovid) Scoping in for loops
by Ovid (Cardinal) on Aug 20, 2000 at 18:34 UTC
    Consider the for loop in the following form: 
    my $i;

for $i (1..8) {
    print "$i\n";
}

    [download]
    In this case, $i is not being assigned the values 1 through 8. Instead, it becomes an alias to the values being iterated over. That's why so many Perl newbies use a construction like this to iterate over an array and then wonder why their array changed at the end. As for the scoping issue, page 100 of Programming Perl states that the variable immediately after the for|foreach in these loops (or $_ if not specified) is implicitly local to the loop and will regain its former value after exiting the loop. Now look at the following snippet: 
    for $i (1..8) {
    push( @ref_list, sub { print "    \$i == $i\n" } );
}

    [download]
    Because $i is actually an alias to the variables being iterated over, the second $i in \$i == $i is pointing to the memory location where the appropriate value in the for loop is being stored. If the value at that memory location could be changed, your anonymous subroutine would print a different value. This is weird, but it works: 
    my ( @ref_list, $i, $j );

$j = 10;
for $i ($j) {
    push( @ref_list, sub { print "    \$i == $i\n" } );
}
&{$ref_list[0]};
$j = 20;
&{$ref_list[0]};

    [download]
    The above will print: 
    
    $i == 10
    $i == 20

    The C-style for ($i; $i<=8; $i++) {} does not have this benefit. Here, you've changed the value of $i, period. Unless you've exited early or done something funky with $i's value, it will be equal to 9 at the end of this loop. Consider the following snippet: 

    for ($i = 1; $i <= 8; $i++ ) { 
    push( @ref_list, sub { print "    \$i == $i\n" } );
}

    [download]
    There is no "implicit alias" here, so the second $i simply points to $i.

    Hope this helps.

    Cheers,
    Ovid

    Update: While I got the point across, I have to say that this is one of the most poorly written posts I have done. Ugh. No more posting before coffee!!!

[reply]
[d/l]
[select]
      Thank you very very much for the explanation, Ovid. I see now where I was going wrong. I'm not sure why I was expecting the anon sub ref to be using a copy of the lexical, but now I am understanding "deep binding" better. Maybe I should blame it on the word "magically" in the FAQ. ;) 
      Paris Sinclair    |    4a75737420416e6f74686572
pariss@efn.org    |    205065726c204861636b6572
http://sinclairinternetwork.com
[reply]
RE (tilly) 1: for loops, closures
by tilly (Archbishop) on Aug 20, 2000 at 18:50 UTC
    Ovid already got it, just some details.

    Yes, I have searched for "for" in the documentation. I found it easier to search for "foreach" in "perlsyn" in the end. :-)

    Your problem really is that you need to think carefully about what variables are created where and when. You can get lucky (as you did with looping over a list) and have it work by accident, but the safest approach is to make absolutely sure that you have created a new variable each time with "my".

    Incidentally there is a pretty good discussion of closures in the perltoot page. (And scattered through the other documentation. Try "perldoc -l perl", go to that directory, and start grepping for "closure". :-)

[reply]
      You can get lucky (as you did with looping over a list) and have it work by accident, but the safest approach is to make absolutely sure that you have created a new variable each time with "my".

      Yes, but using my in the normal place does nothing at all solve the problem. As for "getting lucky," just because because there is a structure I experimented with and failed, doesn't mean the way I did it that did work worked by luck. 

      Paris Sinclair    |    4a75737420416e6f74686572
pariss@efn.org    |    205065726c204861636b6572
http://sinclairinternetwork.com
[reply]
        In case it isn't obvious, I am not guessing about why what you did worked and what the gotchas are. Consider the following code: 
        my @list = 1..8;
my @ref_list;

foreach my $i (@list) {
    push( @ref_list, sub { print "    \$i == $i\n" } );
}

foreach my $i (@list) {
    $i = "gone";
}

print "for SCALAR LIST\n";
while ( $code_ref = pop @ref_list ) {
    &$code_ref;
}

        [download]
        Do you see what happened there?

        Nope. It was not an accident that your code worked. Perl worked exactly as documented.

        But if you code that way without understanding why it works, then you will get bitten eventually. I am saying that because I have been there, done that, and bear the scars. Else I wouldn't care enough to offer advice.

        Incidentally you would be well advised to stop using my as if it was a global declaration. What you call "in the normal way" I call "ill-advised". See RE (3): BrainPain-Help for my attempt to explain why. See how I used it above, moving the "my" as close to the initialization of the variable as possible? I do things that way on purpose.

        A final note. Here is what I meant by making absolutely sure that you create a new variable each time: 

        for (my $i = 1; $i <= 8; ++$i) {
    my $really_private = $i;
    push( @ref_list, sub { print "\t$really_private\n" } );
}

$_->() foreach @ref_list;

        [download]
        See? I am explicitly using a variable scoped to be private per instance of the loop. If this appears cumbersome, it makes more sense in a constructor. Take this simple example: 
        sub ret_simple_formatter {
    my $fmt = shift;
    return sub {sprintf($fmt, shift);};
}

        [download]
        Call that in a loop and it just works perfectly. :-)

        BTW I have used almost exactly that function while developing code to set up a hash of field handlers. As the code matured, of course, most of the fields got better formatters than that. But it worked nicely for a first pass...

[reply]
[d/l]
[select]
Re: for loops
by jeorgen (Pilgrim) on Aug 20, 2000 at 15:39 UTC
    This is just a hunch, but could it be that the $i in the last example is treated as a reference at the end, so that the sub is not evaluated until in the last block. So it is evaluated to the last value of $i, the value that triggered the exit from the construction loop.

    Maybe this should be fixed with scoping $i so it can't be reached by the last block?

    update:This is a bit silly, but this is the way I solved it when I had the same problem ("my $o" in second last block):

    my ( @ref_list, $code_ref, $i );

#pass one
#for SCALAR LIST
for $i (1..8) {
    push( @ref_list, sub { print "    \$i == $i\n" } );
}

print "for SCALAR LIST\n";
while ( $code_ref = pop @ref_list ) {
    &$code_ref;
}

#pass 2
#for INIT ; TEST ; INCREMENT
for (  $i = 1; $i <= 8; $i++ ) { 
    my $o;
    $o=$i;
    push( @ref_list, sub { print "    \$i == $o\n" } );
}

print "for (;;)\n";
while ( $code_ref = pop @ref_list ) {
    &$code_ref;
}

    [download]
    /jeorgen
[reply]
[d/l]

      Excellent! Okay, but, why does it work? Does that mean that for (my $i;;) gives my a ref to a lexical variable instead of the actual variable? Hm, or a ref to a copy of whatever is in the init part? That doesn't seem to be it, because moving my $i before the for (;;) doesn't fix it, either.

      Can anybody explain the scoping differences between these lines: 

      for my $i (1) { print $i }

for ( my $i = 1; $i; $i--) { print $i }

for ( my $i = 1; $i; $i--) { my $i = $i; print $i }

      [download]
      In all three, $i disapears after the closing brace. So, why does the middle one fail with closures? 
      Paris Sinclair    |    4a75737420416e6f74686572
pariss@efn.org    |    205065726c204861636b6572
http://sinclairinternetwork.com
[reply]
[d/l]
        In case my other reply didn't make it clear.

        The first case half-way works. On each pass through the loop you get an alias to a different element of the list. Therefore the different iterations are different variables, but they are still the same as what is in the list so this is dangerous.

        The second case simply breaks since you create one variable and modify it during the loop. Think of closures as working by reference and you see the issue - you get a bunch of references to the same variable so all of them point at the same value in the end. (The last value during the loop.)

        The third case works because inside the loop you explicitly create a new variable which is completely private to that iteration of the loop. This not only means that your attempted closures are different for each iteration, it means that there is nothing still out there which can mess with your private closure by reference.

        Make sense?

[reply]

Back to Seekers of Perl Wisdom