roboticus has asked for the wisdom of the Perl Monks concerning the following question:

Hello, all:

A few weeks ago, I was working on a little code and came up with a situation I didn't expect to see. It's breaking my mental model of Perl, so I'd appreciate an explanation. The situation: I have a global variable that I'm using as a loop index in a for statement. Inside the for statement, the variable takes on the expected values. But calling a subroutine that happens to use that value leaves that variable uninitialized. I've looked around a bit (such as reading perldoc perlsyn, but can't find anything in there that explains this behavior.

I've distilled the code into a simple example:

$ cat unexpected.pl #!env perl use strict; use warnings; my $S4; sub X { print "<$S4>\n"; return $S4; } for $S4 (1 .. 2) { print "A: <$S4> <", X(), ">\n"; } $ perl unexpected.pl Use of uninitialized value $S4 in concatenation (.) or string at unexp +ected.pl line 8. <> Use of uninitialized value in print at unexpected.pl line 13. A: <1> <> Use of uninitialized value $S4 in concatenation (.) or string at unexp +ected.pl line 8. <> Use of uninitialized value in print at unexpected.pl line 13. A: <2> <>

If I change the "my $S4;" to "our $S4;" it works as I'd expect it to.

It's hard to imagine that it's a perl bug, as I'd expect someone to have noticed something like this, but it's so contrary to my mental model of Perl that it's nagging the back of my brain.

Just in case it matters:

$ perl --version This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-cy +gwin-threads-multi (with 7 registered patches, see perl -V for more detail) Copyright 1987-2017, Larry Wall Perl may be copied only under the terms of either the Artistic License + or the GNU General Public License, which may be found in the Perl 5 source ki +t. Complete documentation for Perl, including FAQ lists, should be found +on this system using "man perl" or "perldoc perl". If you have access to + the Internet, point your browser at http://www.perl.org/, the Perl Home Pa +ge.

Thanks in advance,

...roboticus

When your only tool is a hammer, all problems look like your thumb.

Replies are listed 'Best First'.
Re: Why is it uninitialized?
by Laurent_R (Canon) on Dec 20, 2017 at 18:50 UTC
    From the documentation on the foreach loop (https://perldoc.perl.org/perlsyn.html#Foreach-Loops):
    The foreach loop iterates over a normal list value and sets the scalar variable VAR to be each element of the list in turn. If the variable is preceded with the keyword my, then it is lexically scoped, and is therefore visible only within the loop. Otherwise, the variable is implicitly local to the loop and regains its former value upon exiting the loop. If the variable was previously declared with my, it uses that variable instead of the global one, but it's still localized to the loop. This implicit localization occurs only in a foreach loop.

    The foreach keyword is actually a synonym for the for keyword, so you can use either. If VAR is omitted, $_ is set to each value.

    Update: added some formatting in the above quote.

      Except the effect of local also applies to the called subroutines,as shown by the our version. perlsub says:

      A local modifies its listed variables to be "local" to the enclosing block, eval, or do FILE --and to any subroutine called from within that block. A local just gives temporary values to global (meaning package)
      So while local is supposed to work across calls to subroutines, it's also supposed to work on variables which are present in the symbols table. I wouldn't expect a closure to keep the name of the variable, or whatever is used to make the localization work

        Yeah, Eily, I understand your point.

        I am not quite sure, though, whether the sentence the variable is implicitly local to the loop and regains its former value upon exiting the loop is to be construed as meaning that it works as if the loop variable is localized with the local keyword.

      Presumably the declaration with our prevents the implicit localization within the loop.

      poj
Re: Why is it uninitialized?
by hexcoder (Curate) on Dec 20, 2017 at 21:03 UTC
    In [Perl Best Practices] from Damian Conway, there is an entry 'Non-Lexical Loop Iterators' in chapter 6 warning about this construct.

    Without my, Perl does not use the variable $S4 for the loop, but instead uses a new lexically variable also named $S4. Thats why the first variable $S4 is uninitialized in sub X.

    So your code behaves like this:

    #!env perl use strict; use warnings; my $S4; sub X { print "<$S4>\n"; return $S4; } for my $other_variable_also_named_S4 (1 .. 2) { print "A: <$other_variable_also_named_S4> <", X(), ">\n"; }

    Conway warns to use this construct since its 'behaviour is contrary to all reasonable expectation'.

    BTW: perlcritic warns about it too.

      BTW: perlcritic warns about it too.

      But, then, that's hardly surprising, given that "Perl::Critic is distributed with a number of Perl::Critic::Policy modules that attempt to enforce various coding guidelines. Most Policy modules are based on Damian Conway's book Perl Best Practices."

Re: Why is it uninitialized?
by Eily (Monsignor) on Dec 20, 2017 at 18:10 UTC

    I'm as surprised as you are. Only thing I see is that for doesn't *set* the loop variable AFAIK, but it turns it into an alias (if you had had other variables in your list, no new value would have been created, but $S4 would have pointed to the existing scalars). So it looks like the closure still looks at the previous scalar, bypassing the aliasing mechanism.

Re: Why is it uninitialized?
by pryrt (Abbot) on Dec 20, 2017 at 18:15 UTC

    I believe you've inadvertantly created something akin to a closure (except it's not anonymous), and perl is using the current value (undef) of the lexical variable $S4 when the subroutine is defined. The variable needs to have package-level scope (which our $S4 gives you; aka "global") in order to be "live" in the subroutine. Or, better, pass it as a parameter to the function.

    Some of this is hinted at in perlsub, search for the first instance of "closure", and perlref. You can read those statements in the docs without realizing the implications, until you actually run across a bug that helps it solidify.

      something akin to a closure (except it's not anonymous)
      In perl, a closure doesn't have to be an anonymous sub; any sub that uses lexical variables which are declared in a scope outside that sub, closes over those vars at the time the sub is created. Anonymous subs are usually more interesting, since multiple instances of the sub are often created at runtime, thus capturing multiple instances of outer lexicals. Conversely, named subs are created just once, at compile time, thus just capturing the first instance of each lexical.

      Dave.

      Although this is almost slightly off-topic, please note that a closure doesn't have to be anonymous (but closures are quite commonly anonymous).

        ++Laurent_R ++dave_the_m: yeah, I didn't originally think it had to be anonymous, but perlref said "Closure is a notion out of the Lisp world that says if you define an anonymous function in a particular lexical context, it pretends to run in that context even when it's called outside the context." Maybe I'm just misinterpreting that (maybe always anon in Lisp? dunno). Or maybe the doc wasn't worded as carefully as it could have been. Thanks for the clarification.

Re: Why is it uninitialized?
by LanX (Saint) on Dec 21, 2017 at 00:30 UTC
    We had several similar discussions in my years here, but unfortunately I'm not able to find them now.(wait°)

    Point is IIRC - like others already said - that "localizing" the lexical happens by binding a new place to the symbol S4.

    Localizing isn't done like in local (safe/restore) with package vars on the same place.

    I'm not sure, but I seem to remember that the implementation of foreach involves an extra "frame" with an own lexpad.

    The whole mess is probably a side effect of the late introduction of lexicals after Perl 4.

    Best solution for your problem is to use another symbol for the loop and to copy the value to the closure var.

    for my $x (1 .. 2) { my $S4 =$x; print "A: <$S4> <", X(), ">\n"; }

    > It's breaking my mental model of Perl, 

    That's because Perl is kind of broken here ...

    ... or to phrase it better, foreach with its aliasing is a very magic beast inside Perl.

    update

    °) for instance How do closures and variable scope (my,our,local) interact in perl?

    Cheers Rolf
    (addicted to the Perl Programming Language and ☆☆☆☆ :)
    Wikisyntax for the Monastery

Re: Why is it uninitialized?
by hdb (Monsignor) on Dec 21, 2017 at 09:03 UTC

    I have to admit ignorance of what really happens inside Perl but a small experiment sheds a bit of light on the situation:

    use strict; use warnings; my $S4 = "aaa"; sub X { print "<$S4>\n"; return $S4; } for $S4 (1 .. 2) { print "A: <$S4> <", X(), ">\n"; }

    which supports the "closure theory", imho.

    Update: adding to my experiment as follows

    use strict; use warnings; my $S4 = "aaa"; sub X { print "<$S4>\n"; return $S4; } for $S4 (1 .. 2) { print "A: <$S4> <", X(), ">\n"; } $S4 = "bbb"; print "B: <", X(), ">\n";

    which seems to confirm the theory that the loop variable is implicitly localized.