b4swine has asked for the wisdom of the Perl Monks concerning the following question:

I have very strange (quantum-like: looking at a variable causes it to exist with the right value!) behavior, which I am sure must be documented somewhere deep in the perl documentation.

I can't replicate it in any small program, so I will describe the behavior here.

Update: see my reply to myself, where a small program to replicate this behavior is shown

A package I have reads:

... my $var ... sub initialize { ... $var = 'this string'; ... } sub mysub { # print STDERR "var = $var"; print STDERR eval("...$var..."); } BEGIN { initialize() }
gives me: Use of uninitialized value in concatenation (.) or string at (eval 40) line 1. but if I uncomment the print line, everything works fine, and 'this string' prints out with no errors at all!

It is a large, spaghetti of a program, but I have verfied that deleting the one comment character produces this result.

I know that looking at a variable can cause it to get defined, but how can a simple print cause an unitialized variable to get defined with the right value?

Replies are listed 'Best First'.
Re: quantum behavior in perl? (cheap closures)
by tye (Sage) on Aug 18, 2007 at 06:13 UTC

    I believe one part of your code is actually closer to eval('...$var...') (note the single quotes). And, yes, this is likely a FAQ somewhere.

    A reference to a Perl sub "closes over" any lexical variables that are in scope when the reference to the sub is taken. One reference to a simple, named sub is taken and stored in the package's symbol table ("stash") when the sub is compiled. This is how a simple, named sub can access lexical variables declared in their surrounding scope.

    For the sake of efficiency, these references only "close over" lexical variables that the subroutine makes use of. Since your mysub() does not make any (direct) use of $var, $var isn't closed over. So when you call mysub(), the reconstruction of the scope to include any closed-over lexicals does not provide access to $var.

    If your code really did say eval("...$var..."), then that would get compiled into the equivalent of eval("...".$var."...") so $var would have been directly used by mysub() and so would have been closed over.

    Yes, this quirk is considered unfortunate and so there is a desire to "fix" it. But I'm aware of a plan for how it could be reasonably fixed, so I wouldn't hold my breath. You'll need to work around it by mentioning $var directly in mysub(), though you don't have to print it.

    - tye        

      Yes it was a typo. I do have '...$var...'. This makes sense, but is very unexpected. I have dozens of variables in the eval, so this is a bit annoying. Anyway, now that I know what the problem is, I can deal with it.

      Thanks

Re: quantum behavior in perl?
by chromatic (Archbishop) on Aug 18, 2007 at 06:07 UTC

    That looks like one of the closure bugs that Dave Mitchell fixed for Perl 5.9.x. In short, sometimes Perl doesn't see all references to lexical variables. Doubly-nested subs (an anonymous sub declared within a named sub) were one offender. Your string-eval construct looks like it could be another.

    The solution for Perl 5.8.x and earlier is to refer to the lexical variable in the intervening scope so that Perl can close over it appropriately.

      No, this isn't fixed in 5.9.5 (not in my copy anyway). And I'm not surprised, since parsing the string before eval gets run in order to check for this potential problem would only be a partial solution and could be an expensive one. The case of nested subs is easier to fix so I'm glad to hear that it has been addressed.

      - tye        

        Perl does parse the string, in the case of double quotes. (Actually, it has to parse every constant string in the program to find the closing quote, but that's not precisely what you meant):

        $ perl -MO=Concise my $foo = 'bar'; my $baz = "foo${foo}bar\n"; e <@> leave[1 ref] vKP/REFC ->(end) 1 <0> enter ->2 2 <;> nextstate(main 1 -:1) v ->3 5 <2> sassign vKS/2 ->6 3 <$> const[PV "bar"] s ->4 4 <0> padsv[$foo:1,3] sRM*/LVINTRO ->5 6 <;> nextstate(main 2 -:2) v ->7 d <2> sassign vKS/2 ->e - <1> ex-stringify sK/1 ->c - <0> ex-pushmark s ->7 b <2> concat[t4] sKS/2 ->c 9 <2> concat[t3] sK/2 ->a 7 <$> const[PV "foo"] s ->8 8 <0> padsv[$foo:1,3] s ->9 a <$> const[PV "bar\n"] s ->b c <0> padsv[$baz:2,3] sRM*/LVINTRO ->d - syntax OK
Re: quantum behavior in perl?
by ikegami (Patriarch) on Aug 18, 2007 at 06:16 UTC

    Sounds like a closure problem.

    { my $var = 'this string'; sub test1 { print("var = $var\n"); } sub test2 { eval(' print("var = $var\n"); '); } sub test3 { $var; eval(' print("var = $var\n"); '); } } test1(); # var = this string test2(); # var = (undef) test3(); # var = this string

    Perl doesn't know that test2 will need $var, so it doesn't capture $var.

    Is your my variable inside curlies? Remember that mod_perl Registry scripts are placed inside a function and therefore inside curlies.

    Update: Exiting any scope will cause closures and thus the problem. Curlies create scopes, but so do do, require and use as tye points out in his reply. The following will also exhibit the problem:

    # Module.pm pacakge Module; my $var = 'this string'; sub test1 { print("var = $var\n"); } sub test2 { eval(' print("var = $var\n"); '); } sub test3 { $var; eval(' print("var = $var\n"); '); }
    # script.pl use Module; Module::test1(); # var = this string Module::test2(); # var = (undef) Module::test3(); # var = this string

      "Curlies" aren't required for this problem. This is not the "will not stay shared problem" (and more than "curlies" is required for that; the my statement has to have the potential to be run more than once -- at least, that was my understanding).

      All that is required for this problem is for the scope in which the lexical was declared to be left before the function is called (I didn't previously realize that even this was required). This can happen if the lexical and sub are in a module, for example. The original node says "A package I have", so a separate file such as a module seems likely; therefore curlies likely don't matter.

      - tye        

Re: quantum behavior in perl?
by bobf (Monsignor) on Aug 18, 2007 at 06:35 UTC

    Update: Wow, it took me a long time to test and compose this (there weren't any replies when I started). The other replies indicate a closure problem, so I may be barking up the wrong tree here.

    Update 2: Yup. Wrong tree. Wrong yard, even. For some reason (I'll blame fatigue) I completely missed the fact that the ellipses in the eval meant "some code deleted here" (even though I understood that perfectly well in the rest of the OP) rather than literal dot-dot-dot's as part of the print statement. Duh. :-)


    From eval for eval EXPR:

    the return value of EXPR is parsed and executed as if it were a little Perl program
    and later, as part of an example, we are told that eval "$x"; "run[s] the code contained in the variable $x".

    I'm going to jump to the end in an attempt to explain what I think might be going on. Your example has $var = 'this string', but since eval is running code it might be easier to follow what's going on if we changed that to something else, say $var = 'print "hi\n"'. I also added a call to mysub() and simplified that routine.

    my $var; mysub(); sub initialize { $var = 'print "hi\n"'; print "in initialize: $var\n"; } sub mysub { print "in mysub: $var\n"; } BEGIN { initialize() }
    This prints:
    in initialize: print "hi\n" in mysub: print "hi\n"
    This example shows that $var is set to the expected value in mysub, so the problem must be with the eval.

    Now change mysub as follows (a simplification of your original eval):

    sub mysub { print "in mysub: $var\n"; print eval("$var"), "\n"; }
    The output is:
    in initialize: print "hi\n" in mysub: print "hi\n" hi 1
    Adding eval("$var") added two lines of output, hi and 1. The first line ("hi") is from evaluating the code in $var (print "hi\n"), and the second is from printing the return value ("1") from the eval.

    Now change mysub to your original code and print $@ in case there are any errors in the eval:

    sub mysub { print "in mysub: $var\n"; print eval("...$var..."), "\n"; print $@ if $@; }
    Output:
    in initialize: print "hi\n" in mysub: print "hi\n" syntax error at (eval 1) line 1, near "..."
    $var is interpolated and the code in the eval is executed, but that code (...print "hi\n"...) contains a syntax error (it's a bit more obvious now, isn't it?).

    Finally, change mysub to:

    sub mysub { print "in mysub: $var\n"; print eval("$var"), "\n"; }
    (you can also remove the quotes around $var in the eval). Output:
    in initialize: print "hi\n" in mysub: print "hi\n" hi 1
    which behaves as we would expect.

    In summary, the problem had nothing to do with initializing the variable, it was due to the fact that the ellipses in your print statement created syntactically invalid code. Moral of the story: check $@ after an eval. :-)

      I have not read your reply in detail, but I did check $@, and there was no error in the eval. Especially since the same eval works after putting in a print statement before it.

      tye gave a good pointer. It was related to lexicals. Once I tried 'use vars' for my variables, instead of 'my', everything is now working.

      Thanks for your detailed experimentation. I will read it.

Re: quantum behavior in perl?
by b4swine (Pilgrim) on Aug 18, 2007 at 11:15 UTC
    Here is a small program that displays the exact problem I encountered. It has to have at least two files to display this behavior.

    The first is a file I call temp.pl which contains

    use temp1; temp1::init(); print "Calling work1: "; temp1::work1('$var'); print "\nCalling work2: "; temp1::work2('$var');

    The second file is a package temp1.pm which contains

    package temp1; my $var; sub init { $var = 1 } sub work1 { print eval $_[0]; } sub work2 { print eval $_[0]; $var = $var; } 1;

    The resulting output is:

    Calling work1: Calling work2: 1

    What is really impressive is that the print in &work2 works even though the modification is after the print statement! This is beyond quantum.

    If I change the my $var to our $var everything works.

    tye explains that perl, not seeing any mention of the lexical $var in &work1, assumes that &work1 does not use it, so the context of the eval does not define that variable. Doing the 'right thing' would be too expensive.

    This is clearly an unsafe optimization, so it appears that some warning is needed. The documentation about the eval function mentions nothing of the sort. Clearly having compiler warnings for every instance of eval seems ridiculous, but I feel that something about this needs to be in the main documentation about the eval function.

      The documentation about the eval function mentions nothing of the sort. Clearly having compiler warnings for every instance of eval seems ridiculous, but I feel that something about this needs to be in the main documentation about the eval function.
      Patches welcome.

      Your example is actually working with two completely different scopes. Consider this:

      use temp1; use warnings; use strict; temp1::init(); use temp1; use warnings; use strict; temp1::init(); print "Calling work1: " . temp1::work1( '$var = 3' ) . "\n"; print "Calling work2: " . temp1::work2( '$var++' ) . "\n"; print "Calling work3: " . temp1::work3( '$var++' ) . "\n"; print "Work3 again: " . temp1::work3( '++$var' ) . "\n"; print "Calling work4: " . temp1::work4( '$var++' ) . "\n";
      package temp1; use strict; use warnings; my $var; sub init { $var = 101 } sub work1 { return eval shift; } sub work2 { my $foo = shift; return eval $foo; $var } sub work3 { return eval $_[0]; } sub work4 { eval $var; } 1;
      The output is:
      Calling work1: 3 Calling work2: 101 Calling work3: 3 Work3 again: 5 Calling work4: 102