Search the Monestary using super search for more info.

-- Randal L. Schwartz, Perl hacker

I'm sorry if this may seem like a stupid question. I'm not very familiar with Perl's innards (except for a trip or two into the bizarre and surreal realm of XS), and I can't really say I understand why the whole variable referencing business is so messy, with the symbol tables and the my()s and the locals() and all. But I've implemented my share of programming languages, functional and otherwise, and closures have never been a problem.

So, is there something special about Perl's environments which makes lexical scoping a problem? While we're at it, is there a good reason why my() isn't the default inside subs?

-- Kaufmann

It's not a problem, it's a good idea.

Lexical scoping is a very good thing because it means that when you use a module written by someone else, or when more than three or four people are working on a project at a time, a variable in one place won't clobber a variable somewhere else with the same name.

I won't say that the innards aren't a little messy (and I haven't read the regex code, so I don't have to roll for Sanity), but the way it works makes sense, and fits the Do What You Mean principle fairly well.

As for why my isn't the default in subroutines, there are two good reasons. First, it would have broken backwards compatibility between Perl 4 and Perl 5. The goal there was to minimize breakage. Second, it's the same reason 'use strict' isn't always on by default -- sometimes, quick and dirty scripts can use global variables and no one cares.

I wasn't there, but my understanding is simply history.

Through Perl 4 Perl had neither lexical scope nor real references. It's scoping mechanisms were package namespaces and dynamic scope through local.

With Perl 5 both of those were added. But they needed backwards compatibility. Therefore they could not by default localize variables in subs, nor could they change local's behaviour. And since the working implementation all used the symbol table (and therefore gave the wrong semantics), a lot of reimplementation and fixing needed to happen. Plus some thought needed to be given to the semantic issues of having so many different (and incompatible) kinds of scope.

Which means that over the 5.* series you have seen constant extension of what can be lexical and how much it is used. For instance from 5.003 to 5.004 they added lexical loop variables, and 5.6 adds lexically scoped declarations of access to global variables.

Remember that Perl is still at its heart a hack written to avoid writing a real report generator that has gone seriously astray. :-)

It seems to me that "my $x" has gone out of scope by the time the anon sub runs...

It has.

is the value still around because there remains a ref to it?

Yes.

If so, what is doing the ref... the anon sub itself?

No, not in my book. The reference to the anonymous sub is a special type of "code ref" called a closure (yes, we all knew that by now). This closure (the code ref) is what holds a reference to the lexical (and not the anonymous sub which is separate from the reference in my book -- especially since the anonymous sub is not recompiled every time you take another reference to it).

So $h holds (something that contains) a reference to the $x that was set to "Howdy" while $g holds a reference to the $x that was set to "Greetings".

Make $x an object and you can see that destroying $h triggers a destructor, etc:

#!/usr/bin/perl -w
use strict;

sub Obit::new {
    my( $this, $ref )= @_;
    my $desc= "$ref";
    $desc .= " (${$ref})"   if  UNIVERSAL::isa( $ref, "SCALAR" );
    warn "$desc is born.\n";
    return bless $ref, $this;
}
sub Obit::DESTROY {
    my $self= shift;
    my $desc= "$self";
    $desc .= " (${$self})"   if  $self->isa("SCALAR");
    warn "$desc has died.\n";
}

sub newprint {
  my $x= Obit->new( \shift );
  return sub { my $y= shift; print "${$x}, $y!\n"; };
}

$|= 1;
{
    my $h= Obit->new( newprint("Howdy") );
    warn "About to destroy \$h.\n";
}
warn "\$h is no more.\n";
{
    my $g= Obit->new( newprint("Greetings") );
    warn "About to destroy \$g.\n";
}
warn "\$g is no more.\n";
__END__
SCALAR(0x1bbefc0) (Howdy) is born.
CODE(0x1bbf074) is born.
About to destroy $h.
Obit=CODE(0x1bbf074) has died.
Obit=SCALAR(0x1bbefc0) (Howdy) has died.
$h is no more.
SCALAR(0x1bb50d4) (Greetings) is born.
CODE(0x1bbf074) is born.
About to destroy $g.
Obit=CODE(0x1bbf074) has died.
Obit=SCALAR(0x1bb50d4) (Greetings) has died.
$g is no more.
[download]

We talk different. :-)

I am curious. I think of anonymous subs in terms of the variable that is holding them, not the function that created them. So if I have 10 variables in an array that each hold a code-ref, I would say that I have 10 anon subs, and not "3 copies of this one, 2 of that, and 5 of the other". I talk about passing anon subs to functions, etc, and this only makes sense to me if I think that way.

Besides which to me using anon subs is a way of hiding information. I use them because I explicitly don't want to and don't think this code should care about where stuff came from. So the constructor is at best irrelevant so long as the function does what it is supposed to!

So how about others? Are you with Tye in thinking about the internals of what gets compiled etc, or with me in thinking about anon functions in terms of the reference you can see in your variables?

The question was about internals (who holds the ref), so the answer was oriented that way as well.

The reason that I so strongly made the distinction between the function and the reference to it, is because I wanted to stress that the reference (to the anonymous sub) going out of scope destroys the reference (to the lexical variable of the closure).

And I also pass anonymous subs to functions. I pass them by reference. ;-)

Note that Perl agrees with me:

  CODE(0x1bbf074) is born.
[...]
  CODE(0x1bbf074) is born.
[download]

both closures are references to the same anonymous function.

Now if there was a way to do undef( &anon ) then I could make the point even stronger. It'd also help if there was a way to have the subroutine code hold a reference to an Obit object so I could show it being destroyed when the (compiled code of the) anonymous function is destroyed. But I don't know how to do either of those, yet (but I have an idea of how I might be able to do one of those so I may post a follow-up about this in a bit).

I will certainly be "sloppy" and talk about an "anonymous sub" when I more precisely mean a "ref to anon sub". There is no problem with that. If I come up with any cases where the distinction really matters (which appears to be quite difficult), then I'd quibble with such "sloppy" usage when specifically discussing these unusual cases.

Note that having two references to the same hash and being sloppy and talking about having two hashes is likely to get you into a mess of trouble if you modify both of your hashes, for example. Since you can't modify code via the code ref, I vote for being "sloppy" most of the time.

        - tye (but my friends call me "Tye")

The idea of lexical scope is that the scope of the variable is defined by the text, it appears in, and its value is bound to that at run-time. So the scope of $x is inside of newprint (including the anon sub), and when you run newprint you get an anonymous function, which sees $x as being the value that it had on the invocation that you created it with.

So run newprint twice and you get two anonymous functions which are using different $x variables. That was just a demo. There really is no limit and in the link I gave you I create, oh, huge scads of copies of very similar functions that way. :-)

Thank you for a spark of enlightenment.

=Blue
...you might be eaten by a grue...