haj has asked for the wisdom of the Perl Monks concerning the following question:

I guess we have all been there: You make an "obviously trivial" change ... and the code fails to behave. This is a stripped down version of the "good" code:

use 5.030;
use warnings;
my @subrefs = ();
for my $i (1..3) {
    push @subrefs, sub { print $i; };
}
for (@subrefs) {
    $_->();
}

[download]

This prints 123, as I would expect.

The "trivial" change:

use 5.030;
use warnings;
my @subrefs = ();
my $i = 0;
for (1..3) {
    $i++;
    push @subrefs, sub { print $i; };
}
for (@subrefs) {
    $_->();
}

[download]

This prints 333. WTF?

It seems the anonymous subroutine is not built with the value of $i, but with a reference to $i. I would not have expected this. Is this a bug, a quirk, or something which can be found documented somewhere?

Edited to add:

I wrote that bad code BC (Before Coffee), remembering that I had done something very similar before. I have now dug that up:

use 5.030;
use warnings;
my @subrefs;
my $i = 0;
for (1..3) {
    push @subrefs, sub { print ++$i };
}
for (@subrefs) {
    $_->();
}

[download]

This prints 123. The difference is that the increment is done when the subroutine is called, not when the subs are declared and pushed to the array. In BC state I failed to see that difference.

Replies are listed 'Best First'.
Re: Closures and scope of lexicals
by ysth (Canon) on Oct 30, 2024 at 18:46 UTC

    In the for my ... case, the loop lexical is implicitly scoped to inside the for body. At the point it reaches the end of the block, the lexical goes out of scope and perl notes that there's an external reference to it and allocates a new one. (This is true for all lexicals and blocks, not specific to for.)

    In the second case, the lexical never goes out of scope, so all closures reference the same scalar, just as if you did push @subrefs, sub { print $i }, sub { print $i }; in the "good" code it would produce 112233.

    --
    A math joke: r = | |csc(θ)|+|sec(θ)| |-| |csc(θ)|-|sec(θ)| |
[reply]
[d/l]
[select]

      Yeah, thanks for the explanation. The key is "all closures reference the same scalar". Somehow I assumed that the closure would use the current value of the scalar (in a new slot). Thanks also to LanX and Hippo for their (similar) explanations!

[reply]
        I assumed that the closure would use the current value of the scalar

        Definitely not. If that were the case, then calling it "binding" (or "closing over") would make no sense. The point is that it binds to the variable in the enclosing scope. So both the sub and any other code with access to that variable can change it. 

        sub make_private_var_with_accessors
{
    my $var;
    (
        sub { my $val = $var; $val }, # getter
        sub { $var = shift } # setter
    )
}

my($get,$set) = make_private_var_with_accessors();
$set->(42);
print "=".$get->()."\n";

        [download]
        And it's but a hop and a skip to turn that into a POPO class: 
        {
    package PrivateVariable;
    sub new
    {
        my $pkg = shift;
        my $var;
        bless {
            get => sub { my $val = $var; $val },
            set => sub { $var = shift },
        }, $pkg
    }
    sub get { $_[0]->{get}->() }
    sub set { $_[0]->{set}->($_[1]) }
}

my $pv = new PrivateVariable;
$pv->set(666);
print "=".$pv->get()."\n";

        [download]
        Today's latest and greatest software contains tomorrow's zero day exploits.
[reply]
[d/l]
[select]
        You can also create multiple closures which access the same but otherwise private variables.

        Closure-subs have a lot in common with object-methods and the closure-vars act like instance variables.

        Again, please have a look at the HOP book, you won't regret it.

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        see Wikisyntax for the Monastery

[reply]
Re: Closures and scope of lexicals
by hippo (Archbishop) on Oct 30, 2024 at 18:12 UTC

    You might think of it as a quirk but it doesn't look like a bug to me.

    In the first instance the anonymous subs are closing over the local $i which is reset every time through the loop - ie. it is a different variable on each iteration. That is how the different values persist. In the second instance, there is no local $i, just the outer global one and so the sub closes over that and clearly its value changes over time but all three subs are now essentially the same.

    🦛

[reply]
Re: Closures and scope of lexicals
by LanX (Saint) on Oct 30, 2024 at 18:14 UTC
    That's textbook closure behavior. The reference is stored in the (scratch)pad of the sub.

    Compare PadWalker

    A closure is a sub plus the variables in the enclosing "hull" at point of definition.

    Another good reference is Higher Order Perl

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    see Wikisyntax for the Monastery

[reply]
Re: Closures and scope of lexicals
by ikegami (Patriarch) on Oct 31, 2024 at 10:13 UTC

    The variable is captured (i.e. kept around), not the value.

    I would not have expected this.

    Weird. You read $i, but expect to get the value of something other than $i?

    is this [...] something which can be found documented somewhere?

    Closures are documented in perlsub. "If something more permanent is still aware of the lexical, it will stick around."

    This means you can access the lexical beyond when it would normally be destroyed for going out of scope (which doesn't actually happen in your second snippet).

    Reading the variable still gives its current value as usual.

    is this a bug [...]?

    Absolutely not.

    Imagine if the following module didn't work because $i didn't get captured by get and set: 

    package Foo;
my $i;
sub get { $i }
sub set { $i = shift; }
1

    [download]
    (These are true closures like your first snippet but unlike your second since $i goes out of scope at the end of the file before get and set are called.)

    is this [...] a quirk [...]?

    Absolutely not.

    Same in JavaScript. 

    const subrefs = [];
let i = 0;

for ( let j = 3; j--; ) {   
   i++;
   subrefs.push( function(){ console.log( i ); } );
}

for ( const subref of subrefs ) {
   subref();
}

    [download]
    3
3
3

    [download]

    Same in C# 

    List<Action> subrefs = new();
int i = 0;

for (int j = 0; j < 3; ++j )
{
    i++;
    subrefs.Add( () => Console.WriteLine( i ) );
}

foreach ( var subref in subrefs ) {
   subref();
}

    [download]
    3
3
3

    [download]

    Same in Python 

    subrefs = []
i = 0

for j in range(3):
   i = i + 1
   def subref():
      print( i )

   subrefs.append( subref )

for subref in subrefs:
   subref()

    [download]
    3
3
3

    [download]
[reply]
[d/l]
[select]
      Closures are documented in perlsub. "If something more permanent is still aware of the lexical, it will stick around."

      That's the relevant pointer. Thanks! I should have found that by myself.

[reply]
      Tho python is still different IIRC.

      There is no block scope, variables live in a function or a file.

      In order to create a list of closure functions in a loop closing over different variables - like in the OP's first example - one would need to call an explicit generator function, having the var in its scope and returning the closure.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      see Wikisyntax for the Monastery

[reply]

        But it's supposed to be file-scoped. $i is scoped to the file in the OP's code. Your suggested change would make the code different than the OP's code.

[reply]
[d/l]

Back to Seekers of Perl Wisdom