Unpredicted late destruction

ferrency has asked for the wisdom of the Perl Monks concerning the following question:

I just came across a behavior in Perl (5.8.0) which I did not predict. When calling a function inside a foreach statement's LIST, any lexicals declared inside that function were not destroyed when I expected them to be destroyed.

Here is a minimal example which demonstrates what I mean:


#!/usr/local/bin/perl
use Fcntl qw(LOCK_EX LOCK_UN LOCK_NB);


# This opens a file in a lexcially scoped file handle, and flocks it.
# It will close and unlock when that variable goes out of scope.
sub fn {
    my $fn = "FOO";
    open my $F, ">$fn";
    flock($F, LOCK_EX) or die "no lock";
    
    return $fn;
}

# In this case, for some reason $F above isn't destroyed until after
# the for loop completes.
for my $fn (fn()) {
    open my $G, $fn;
    flock($G, LOCK_EX|LOCK_NB) or warn "no lock 2";
}
[download]

This warns no lock 2 because the lexical $F inside fn() is not destroyed until after the foreach loop is finished.

Does anyone have a good explanation for why this makes sense? A more complete example is below...

In this example, it warns no lock 3 and no lock 4, but the rest of the cases destroy fn()'s lexical variables when I'd expect.

#!/usr/local/bin/perl
use Fcntl qw(LOCK_EX LOCK_UN LOCK_NB);


# This opens a file in a lexcially scoped file handle, and flocks it.
# It will close and unlock when that variable goes out of scope.
sub fn {
    my $fn = "FOO";
    open my $F, ">$fn";
    flock($F, LOCK_EX) or die "no lock";
    
    return $fn;
}


my $fn = fn();
for ($fn) {
    open my $G, $fn;
    flock($G, LOCK_EX|LOCK_NB) or warn "no lock 2";
}

# In this case, for some reason $F above isn't destroyed until after
# the for loop completes.
for my $fn (fn()) {
    open my $G, $fn;
    flock($G, LOCK_EX|LOCK_NB) or warn "no lock 3";
}

for (my $fn = fn()) {
    open my $G, $fn;
    flock($G, LOCK_EX|LOCK_NB) or warn "no lock 4";
}

while (my $fn = fn()) {
    open my $G, $fn;
    flock($G, LOCK_EX|LOCK_NB) or warn "no lock 5";
    last;
}

open my $G, $fn;
flock($G, LOCK_EX|LOCK_NB) or warn "no lock 6";
[download]

And, to demonstrate that it's not limited to the case of implicitly closing filehandles:


sub FOO::DESTROY {print "DESTROYING FOO\n"}

sub fn {
    my $fn = "FOO";

    my $x = bless \$fn, "FOO";
    return $x;
}

# In this case, for some reason $F above isn't destroyed until after
# the for loop completes.
for my $fn (fn()) {
    print "I got $fn\n";
}
[download]

This prints I fot FOO before DESTROYING FOO.

Thanks for any light you can shed on this one!

Alan

Comment on Unpredicted late destruction Select or Download Code

Replies are listed 'Best First'.
Re: Unpredicted late destruction by Abigail-II (Bishop) on Nov 05, 2003 at 17:06 UTC
The last example is easily explained (except you overloaded the name 'fn' which makes your code harder to read, distracting it from the real problem). You function returns an object, which is a reference to the lexical variable $fn. (Not, not that lexical variable $fn, the other one). $x goes out of scope when leaving fn, but $fn (right, the other one) doesn't, as there's still a reference to it. In the for loop, the reference to $fn (the other one) is aliased to $fn (right, that one). It isn't until the end of the loop that $fn (that one) is no longer an alias for a reference to $fn (not that one, the other one). At that moment, there's no reference to $fh (the other one) anymore, so the object is destroyed. Next time, please pick some none colliding variable names. Abigail	[reply]
Re: Re: Unpredicted late destruction by broquaint (Abbot) on Nov 05, 2003 at 17:55 UTC
Update: ack, this appears to be a pre-5.8.0 bug as it's fine after `5.8.0` e.g `shell> perl5.8.0 pmsopw_304778.pl loop: pre buh-bye: sub buh-bye: sub loop: inside buh-bye: for loop: inside buh-bye: for loop: post` [download] Perhaps this is a better illustration of the problem `sub DESTROY { print "bye: @{$_[0]}\n"; } sub f { my $funcobj = bless["sub"]; return 1; } print "loop: pre\n"; for( f(), f() ) { print "loop: inside\n"; my $forobj = bless ["for"]; } print "loop: post\n"; __output__ loop: pre loop: inside bye: for loop: inside bye: for bye: sub bye: sub loop: post` [download] So `$funcobj`, which one would assume to be GCed upon the exit of `f()`, sticks around until the end of the `for` loop regardless of the fact that nothing is referencing it. HTH `_________ broquaint`	[reply] [d/l] [select]
Re: Re: Re: Unpredicted late destruction by ferrency (Deacon) on Nov 12, 2003 at 15:36 UTC
Thank you for the great example of a minimal failure case! And thanks for determining that this has already been fixed :) I've discovered that the same problem exists for functions called in an `if` expression as well. Adding the following test to the bottom of your script does the wrong thing in 5.6.1 but the right thing in 5.8.0: `print "if: pre\n"; if (f()) { print "if: inside\n"; my $ifobj = bless ["if"]; } print "if: post\n";` [download] Interestingly (to me, anyway), there didn't seem to be a problem with `while()` clauses: `print "while: pre\n"; while (f()) { print "while: inside\n"; my $wobj = bless ["while"]; last; } print "while: post\n";` [download] This does the right thing on 5.6.1. Thanks again! Alan	[reply] [d/l] [select]
Re: Re: Unpredicted late destruction by ferrency (Deacon) on Nov 05, 2003 at 17:51 UTC
You're right- the last example is a straightforward case of things working exactly how I should've expected them to. I think I shouldn't have tried to toss that example together at the last minute as I did, because now that I look at it, it's obvious why it's doing what it's doing: the object still exists. Not only that, when I change "return $x" to "return $fn", $x is destroyed Before the body of the for() loop, which disproves my contention that this is happening to All lexically scoped variables in functions called in a for() LIST. So now I guess I'd just like an idea of why filehandles aren't closed when I'd expect them to be :) Sorry about the variable names. Thanks, Alan	[reply]
Re: Unpredicted late destruction by broquaint (Abbot) on Nov 05, 2003 at 16:59 UTC
My memory is a little scratchy on this subject, so bear with me - this behaviour is due to the fact that lexical variables in the argument list are bound to the scope of the `for` loop, and therefore aren't garbage collected until the loop is exitted. As for the lexicals in the subroutine I think this is due to the subroutine scope also being bound to the `for` loop, hence the late destruction. For a more definitive answer, you may find better luck with the p5p folk. HTH `_________ broquaint`	[reply]