Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

understanding closures

by reasonablekeith (Deacon)
on Sep 23, 2005 at 11:10 UTC ( [id://494466]=perlquestion: print w/replies, xml ) Need Help??

reasonablekeith has asked for the wisdom of the Perl Monks concerning the following question:

Okay, I've been programming in Perl for long enough now, and I really feel I ought to understand closures. So I read through the relevant section of Programming Perl, and check out the the first few google hits, which includes the very useful article Achieving Closure, and I'm sort of there with my understanding, but not quite...

A nice and succinct example of a closure I read through is as follows.

sub count_maker { my $count = shift; return sub {$count++} } my $a_counter1 = count_maker(0); my $a_counter2 = count_maker(4); print $a_counter1->() . "\n"; print $a_counter2->() . "\n"; print $a_counter2->() . "\n"; print $a_counter1->() . "\n"; __OUTOUT__ 0 4 5 1
Pretty cool. I can see that, at the time and scope of the print statement, $count is not defined. I can also accept that the locally scoped variables (relative to the sub ref) are "packaged up" uniquely for each sub ref, but what I really don't understand what it is that triggers this behaviour. Programming Perl has the following to say about Closures

Closure is a notion out of the Lisp world that says if you define an anonymous function in a particular lexical context at a particular moment, it pretends to run in that context even when it's called outside of the context.

Okay, so it's the fact that I'm calling an anonymous sub is it? $count was at the same scope as the sub definition, and therefore gets bundled-up? I can see that, so I had a further play, and came up with this...

for (0..9) { my $count = 3; # just to prove a point print counter(); } do { my $count; sub counter {$count++} } __OUTPUT__ 0123456789
I'm almost certain (*) this constitutes a closure, but it doesn't have any anonymous subroutines anywhere. So what is it about this code that tells perl to keep a copy of $count and not just print 10 zeros?

Talking to the bear, as it were, it has just occured to me that the "do" might be the anonymous function in this example, but I'm still not sure what's _actually_ going on here. I hate using something I don't fully understand, so I'd be very grateful if anyone can nudge my understanding the right direction.

Many Thanks, Rob

* but not actually certain, this may be why I'm having problems :p

---
my name's not Keith, and I'm not reasonable.

Replies are listed 'Best First'.
Re: understanding closures
by Limbic~Region (Chancellor) on Sep 23, 2005 at 12:35 UTC
    reasonablekeith,
    You may want to have a look at Help with the concept of closures. and Closure on Closures. A closure is basically a piece of code that can be executed outside of the scope it was defined, but can still remembers things from that scope. I know you were asking specific questions but I feel those two links will help give you a broader and deeper understanding of the concept.

    Cheers - L~R

      Brilliant, Closure on Closures did it for me. The summing up in particular...

      a closure is a subroutine which wraps around lexical variables that it references from the surrounding lexical scope which subsequently means that the lexical variables that are referenced are not garbage collected when their immediate scope is exited

      This story has a sad end, however. To test my knowledge new-found knowledge, I wrote the following.

      sub make_two_counters { my $count = 0; return sub{ $count++ }, sub{ $count+=10 } } my ($counter_plus_one, $counter_plus_ten) = make_two_counters(); print $counter_plus_one->() . "\n"; print $counter_plus_one->() . "\n"; print $counter_plus_ten->() . "\n"; print $counter_plus_one->() . "\n"; print $counter_plus_one->() . "\n"; __OUTPUT__ 0 1 12 12 13
      I was expecting this to print 0,1,11,12,13 - on the grounds that my two anonymous subs would be saving the same reference to the lexical ($count). But it seems to have merged two of my adds in together!! Help.

      BTW: Thanks for all the replies ++

      ---
      my name's not Keith, and I'm not reasonable.
        reasonablekeith,
        Here is a clue - what happens when you forget about closures and run the following code?
        #!/usr/bin/perl use strict; use warnings; my $cnt = 0; print $cnt++, "\n"; print $cnt++, "\n"; print $cnt += 10, "\n"; print $cnt++, "\n"; print $cnt++, "\n";
        The ++ operator (see perlop) first retrieves the current value and then increments the value when used as post-increment.
        • Start at 0
        • Return 0, make $cnt 1
        • Return 1, make $cnt 2
        • Return 2 + 10, make $cnt 12
        • Return 12, make $cnt 13
        • Return 13, make $cnt 14

        Cheers - L~R

        I was expecting this to print 0,1,11,12,13 - on the grounds that my two anonymous subs would be saving the same reference to the lexical ($count). But it seems to have merged two of my adds in together!! Help.

        Your closures are correct. However, you're confused about something else. $count++ will return the value of $count, then add 1 to it. $count+=10 will add 10 to count, then return the value of $count.

        Try the following:

        my $count = 0; sub make_two_counters { return sub{ $count++ }, sub{ $count+=10 } } my ($counter_plus_one, $counter_plus_ten) = make_two_counters(); print $counter_plus_one->() . " ($count)\n"; print $counter_plus_one->() . " ($count)\n"; print $counter_plus_ten->() . " ($count)\n"; print $counter_plus_one->() . " ($count)\n"; print $counter_plus_one->() . " ($count)\n"; __OUTPUT__ 0 (1) 1 (2) 12 (12) 12 (13) 13 (14)

        Does that help?


        My criteria for good software:
        1. Does it work?
        2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
        They are sharing the same lexical. What confuses you is the fact that you are doing post-increment. The result of $count++ is the old value of $count. Change that line to ++$count, and try again.
Re: understanding closures
by blokhead (Monsignor) on Sep 23, 2005 at 12:36 UTC
    The anonymous form of sub { ... } is like a dynamic subroutine constructor. Every time you get to the statement
    return sub {$count++}
    the Perl interpreter generates a new sub. It also happens to bundle up any lexical variables that the sub uses, which you seem to understand.

    The my $var statement is also like a dynamic variable constructor. Every time you get to a my declaration, a brand new variable is created. Since you construct the closure inside this sub:

    sub count_maker { my $count = shift;
    So every time you call count_maker, a brand new $count variable is created and bundled up in the brand new anonymous sub closure. Then we leave the scope of count_maker, so that version of $count disappears other than its reference in the anonymous sub that is still around.

    There are several things different about your other example. First, the non-anonymous form of sub name { ... } works a little differently. It generates a new sub at compile time, not at runtime when the statement is reached. That's why you can call a named sub that is defined lower down in the file. Also, you have two different $count variables in different scopes. The counter sub is always pointing to the one defined inside the do {} block, not the one in the calling scope. Also, the $count variable that is actually referenced in the sub is never redeclared (the do {} block is never entered more than once).

    Here's another example that might be enlightening:

    { my $count1; ## only one copy of $count1 ever created, so ## it's shared by all closures created below sub count_maker { my $count2 = shift; ## $count2 is a different var each time ## each time we enter count_maker return sub { ($count2++, $count1++) } } } my $c1 = count_maker(1); my $c2 = count_maker(5); print join " ", $c1->(), $/; print join " ", $c2->(), $/; print join " ", $c2->(), $/; print join " ", $c1->(), $/; print join " ", $c1->(), $/; __OUTPUT__ 1 0 5 1 6 2 2 3 3 4

    blokhead

Re: understanding closures
by japhy (Canon) on Sep 23, 2005 at 11:30 UTC
    The purists might not call it a closure, but it operates under the same principle. In your second example, counter() makes a reference to $count, a lexical in the scope of the function. It's the same principle, just without the indirection of a function reference.

    Jeff japhy Pinyan, P.L., P.M., P.O.D, X.S.: Perl, regex, and perl hacker
    How can we ever be the sold short or the cheated, we who for every service have long ago been overpaid? ~~ Meister Eckhart
      Actually, the purists will say that it is a closure. A closure (technically) does not have to be a lambda / anonymous sub / dynamically generated sub.
Re: understanding closures
by shady (Sexton) on Sep 23, 2005 at 11:27 UTC
    Play with it:
    #!/usr/bin/perl use warnings; use strict; package Class::GetSet; sub new { my $class = shift; return bless({@_}, $class) } sub __set { $_[0]->{ $_[1] } = $_[2] } sub __get { return $_[0]->{ $_[1] } } use vars qw($AUTOLOAD); sub AUTOLOAD { no strict 'refs'; if ($AUTOLOAD =~ /^.*::(\w+)$/o && exists $_[0]->__has->{$1}->{'rea +d'} ) { my $attr = $1; *{$AUTOLOAD} = sub { my $s = shift; return ($s->__get( +$attr, @_)) }; goto &$AUTOLOAD; } elsif ($AUTOLOAD =~ /^.*::set_(\w+)$/o && exists $_[0]->__ha +s->{$1}->{'write'} ) { my $attr = $1; *{$AUTOLOAD} = sub { my $s = shift; return ($s->__set( +$attr, @_)) }; goto &$AUTOLOAD; } # subs like DESTROY return if $AUTOLOAD =~ /^.*::[A-Z]+$/; die "Unimplemented $AUTOLOAD"; } + + package Dummy; + + use base qw(Class::GetSet); sub __has { return { foo => { read => 1, write => 1 }, bar => { read = +> 1 } } } + + package main; + + my $d = Dummy->new( foo => 1, bar => 2); print "foo = '". $d->foo ."'\n"; $d->set_foo( 'xxx' ); print "foo = '". $d->foo ."'\n"; + + print "bar = '". $d->bar ."'\n"; $d->set_boo( 'xxx' );
    Example above is for example of using the closures for dynamic generation of methods.
Re: understanding closures
by demerphq (Chancellor) on Sep 23, 2005 at 17:08 UTC

    This code is a little unusual. The do {} is a runtime construct to allow you to put statements where an expression is expected. It has scope, but is not actually block (iirc) in that you can't use next to exit out of it.

    D:\dev>perl -e"do { next };" Can't "next" outside a loop block at -e line 1. D:\dev>perl -e"{ next };"

    The named sub there is a closure, but only in a trivial sense, and you'll probably more likely see people talk about "static" vars in this situation. For instance I would expect to see that code written something like:

    BEGIN { my $count=0; # static sub counter {$count++} } for (0..9) { print counter(); }

    Without the do on there the block is really a block (albeit a magic one), and its clear the $count=0 is only going to be executed once, and its going to occur before counter() is ever used, that is immediately after the block is compiled (because of the BEGIN).

    If you want to make a counter factory then you get into more interesting forms of closures:

    sub make_counter { my $start=shift||0; return sub {$start++}; }
    ---
    $world=~s/war/peace/g

      The named sub there is a closure, but only in a trivial sense

      Thanks demerphq, I understand this now. I can see that the power of closure comes when you're using them dynamically, hence why all the examples show anonymous sub references. However, using these simple examples has really helped my understanding, so ++ all round.

      Rob

      ---
      my name's not Keith, and I'm not reasonable.
Re: understanding closures
by furry_marmot (Pilgrim) on Sep 23, 2005 at 16:52 UTC
    I think what's not being stated explicitly is that in your first example, you've created an anonymous subroutine and then returned a reference to it. Executing the coderef triggers the code, despite the fact that it's in a different lexical context than the coderef. You passed in the value to count_maker(), and the value was passed to the anonymous sub because that sub is in the same lexical context as the $count you shifted in.

    In your second example, my $count is in the lexical context of the do{} loop. You don't have strict turned on, so you can access counter(), but the my $count there overrides your declaration of $count in the for{} loop.

    Closures are subtle, but in the context of what you're trying to do, you should be returning a coderef for it to work. Once you get that, read some of the closure articles again.

Re: understanding closures
by enriirne (Initiate) on Sep 27, 2005 at 14:50 UTC
    A little piece of code which uses two closures, partial application and a higher order function. The poor perl code reflects my few weeks on this language.
    # getRowsLazy :: DB_Handle -> (String -> String -> ()) -> (SQLQuery -> + {':parm1' => value, ...} -> [[DBType, ..., DBType]]) # Closure on $dbh, lazy version and partial application on the first t +wo params # # ex: my $dbh = ...DBI connect... my $getRows = F::getRowsLazy($dbh, sub { myErrFun($[0], $[1], $M +Y_FUN_NAME, $MY_TRACE_LEVEL) } ); # ... # my $iter = $getRows->("select * from device_param where ID=:id", + {':id' => 123456}); # while (my $row = $iter->()) { # print $row->[0] . ", " . $row->[1] . "\n"; # } # sub F::getRowsLazy { my ($dbh, $errFun); return sub { my ($sql,$p) = @_; my $sth = $dbh->prepare($sql) || $errFun->("Error in DBI::prepare" +, $dbh->errstr() ); while (my ($k,$v) = each(%$p)) {$sth->bind_param($k, $v);} $sth->execute() || $errFun->("Error in DBI::execute", $dbh->errstr +() ); my $yield = sub { my $row = $sth->fetchrow_arrayref; if ($row) {return $row;} $sth->finish(); return undef; }; return $yield; # the call to $yield->() returns a referenc +e; doesn't work with array, but I don't know way } };

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://494466]
Approved by marto
Front-paged by Tanktalus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2024-04-20 06:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found