I assume references are a structure containing a counter of refs, and a pointer.

Internally, variables are associated with their own refcounts. This is true of lexicals as well as global (i.e. package) variables. If you have $variable and later assign \$variable to $reference then the refcount associated with $variable increases. The refcount is not associated with $reference. (Unless, of course, there is a reference to the reference floating around somewhere.) A variable's storage is not deallocated until its refcount has gone to zero.

-sauoq
"My two cents aren't worth a dime.";

I assumes lexical refs and dynamic refs are not different, as they are just a means of finding the data and freeing it's space when unused.... What I ment by reference is actually not the referencing part of it, more the resolving to a value. I should have not used the word reference, as it is not correct...

In terms of my question - I guess i need to be more clear - How are lexical variables found via their name, and does the symbol table -> value path work like I assumes?

-nuffin
zz zZ Z Z #!perl

I think you're conflating the two senses of "reference" here. What we call a "variable" is a name for a structure in memory which contains a C pointer which contains the actual location in memory of the variable's value. [1]. So $user = "diotalevi" has two components - the variable "$user" and the value it points to. Normally we don't ever care about this but if you want more on this you should read Gisle Aas' PerlGuts Illustrated, perlguts, broquaint's Of Symbol Tables and Globs and the other handful of man pages followed by the headers specifically hitting sv.h.

Reference variables are a variation on that in that they are still variables in the sense I just mentioned. The difference is that instead of pointing to a C array they point to another variable. Perl keeps track of what sort of things a variable points to via a FLAGS bitmask.

The other difference you seem to hung up on is between package and lexical variables. If you'd read some of the documents I mentioned you'd know that your symbol table is just a hash of reference variables. In that case it's explicitly name->variable. shotgunefx brought up the lexical variables scratch pad. Those AoAs (known as a PADLIST) are pointed to by various code references (glossing on CV for those of you that know better) and while the compiler has the opportunity to convert the uses of the $foo variable from my $foo to a direct usage of the variable in the array (ie, no looking up of the variable by name) some other extensions like PadWalker exploit the fact that the variable's name is kept around (and probably used by things that are beyond my ken) and can go get the various lists of lexical names to create a user-friendly hash. Clearer?

[1] "Number" variables can store their own value without needing to point to somewhere else.

__SIG__
use B;
printf "You are here %08x\n", unpack "L!", unpack "P4", pack
  "L!", B::svref_2object(sub{})->OUTSIDE;
[download]

Scratchpads are assoiciated with { scopes }

Not quite. Scratchpads are associated with subs, not with blocks. Yes, from a language level it looks like there's a scratchpad per block, but under the hood there really isn't.

Thanks Elian, I'm trying to improve me understanding of internals (Just got Embedding and Extending Perl). In the following example where do the lexicals for the bare block live then?

#!/usr/bin/perl
use PadWalker qw(peek_my);
use strict;
use warnings;
use Data::Dumper;

sub peeker { my $l = shift;  print Dumper(peek_my(++$l)) };

my $outmost = "outmost";

{
 my $inner = "outer";
 print peeker(0);
}

print peeker(0);
[download]



-Lee

"To be civilized is to deny one's nature."

Seriously, variables are in two places, as folks have more or less already pointed out. First are the global variables, the ones you can look up by name. They're stored in the %main:: hash, or a hash hanging off of it. Perl's got a handle on the %main:: hash stuck away in the interpreter structure, and can find it whenever it wants.

Subs have a scratchpad associated with them, where the lexicals are stored. A pointer to this structure is stored in the subroutine's internal bits, and when you enter a sub the interpreter instantiates the scratchpad for the sub.

Finding lexicals is actually easy. The compiler knows what lexicals you're using, as they're compile time constant things. When it goes and builds a sub's scratchpad it just stuffs all the unique lexicals for the sub into it. Since it keeps track of which lexical's gone in which slot, when it generates the "fetch a lexical" code for the interpreter, it just fills in which lexical slot the variable needs to come from.

There's nothing particularly revolutionary, or even cutting-edge, about perl's compiler code. You might want to find a book on the fundamentals of compilers (but not the Dragon book! It sucks) and read through it for a good idea of how this stuff all works.

Update: File-scoped lexicals are, interestingly enough, not a special case. Somewhere in the docs there's an off-hand comment about the entire file being wrapped in a set of braces. Well, as far as the compiler's concerned that's true--the whole file is, if there are any file-scoped lexicals, living inside an anonymous subroutine. (Which makes all the subs in a file with file scoped lexicals really a closure) Makes everything work out nicely.

If anyone's really interested in the way things are stored internally, I'd recommend starting in gv.c and going from there. Otherwise just assume it's all magic, which is as good an explanation as any.

If you're wondering how code like:

  sub bar {
    my $foo;
    {
      my $foo;
    }
  }
[download]

can possibly have only one scratchpad, that's easy--perl knows the two $foo slots are different, and makes sure it gets the right one when it goes looking. (When lexicals are accessed they're all accessed by slot number, not by name, so the actual name is pretty irrelevant at runtime)

The only place that this could get interesting is with string eval, but since perl also tracks the lines that a particular lexical is active for, it's not a problem.

book on the fundamentals of compilers (but not the Dragon book! It sucks)

Why? Some people think that it is the "classic compiler text", what are they missing? Is this another holy war I'm not aware of?

Let me clarify that I'm not challenging you (You're writing Parrot for goodness sake.), but I like to understand differences.

The Dragon book is a classic because it was the first book in the field of any significance, not because it's actually any good. (Consider it a classic compiler book in the same way that "Plan 9 from Outer Space" is a classic SciFi movie) Everything you need for a simple compiler is in there, but you'll get it out with far more pain and confusion than the subject warrants. Newer books in the field, such as Modern Compiler Design by Grune, Bal, Jacobs & Langendoen (just off the top of my bookshelf), are much better and far easier to read.

Makeshifts last the longest.