|
ELISHEVA has asked for the wisdom of the Perl Monks concerning the following question:
I have a wierd situation where A holds a reference to B, but B seems to be destroyed before A. I've ruled out the most obvious causes:
- B is being held as a weak reference. This was ruled out by grepping for calls to Scalar::Util's weaken method, but in any case, I wrote the code and don't ever recall using weak references for this class.
- the reference to B was set to undef by earlier code. This was ruled out by testing to see if the reference was defined immediately before calling undef $A.
According to the perldocs (perlobj), an object is destroyed when the last reference to it is removed:
When the last reference to an object goes away, the object is automatically destroyed.
I would take that to mean that if object A is the sole holder of a reference to object B and A goes out of scope [ and has no package or in scope my variables holding a reference, i.e. refcount=0 ] then Perl would walk the object graph, first calling the destructor to A and then calling the destructors for each reference held solely by A. B cannot be destroyed before A because as long as A exists there is at least one reference to B.
This happens both on global destruction (exit) and controlled explicit destruction, i.e. undef $A.
In short, I'm stumped.
The platform is Perl 5.10.0 running on Debian Lenny. The Perl is threaded, but there is no explicit use of threads, so presumably A and B are on the same thread.
Solution
As ikegami helped me verify, despite my setting undef $A the object wasn't getting destroyed until the global destruction phase. As Corion has confirmed (along with ikegami in the cb), during the global destruction phase Perl does not guarantee the order of destruction, so whether A gets destroyed before B or B before A is undetermined. And indeed, that was what I was observing: sometimes B would get destroyed first and sometimes A The remaining question was why was destruction waiting until global destruction. The code boiled down to $A=SomeClass->new; undef $A so the reference count should have been 0. Perl destroys objects immediately when the ref count goes to 0 (also confirmed in cb and below).
Several suggested a hidden circularity that was keeping the ref count from actually going to 0. In the end the problem was not circularity, but canonization/memo-izing. To prevent duplicate creation of the same object a superclass was storing a hash of all previously created objects keyed by creation parameters. Unfortunately this hash was storing strong references when it should have been storing weak references. As a result, object A maintained a ref count of at least 1 until the canonizing hash was destroyed. As the canonizing hash was a package variable, it wasn't destroyed until the global destruction phase. So too object A. As expected, changing the canonizing hash so that it held weak references resolved the problem and allowed object A to be destroyed immediately after undef $A
Many thanks to the monks who helped me better test what was really happening and who spurred my problem solving thinking in new directions.
Update 1: added info about threading:
Update 2: Added solution
Update 3: clarified "goes out of scope" to response to anonymized user 468275's observation that "out of scope" is not necessarily synonymous with ref count = 0
|
Re: sub DESTROY: Strange ordering of object destruction
by Corion (Patriarch) on Mar 03, 2011 at 09:55 UTC
|
Object destruction order is not respected anymore once Global Destruction sets in.
Global destruction happens after your main script "falls off the end" (or calls exit) and all END blocks have executed.
A way I've used to better localize the memory cycle that keeps objects alive until Global Destruction is to explicitly release the "master object" at the last (executed) line of the program:
undef $object;
If that works and everything gets released properly, this means you have a simple cycle and $object is kept alive somehow. There are some cases of closures that keep your lexical
variables alive beyond the main program. One especially nasty case (for me) is:
my $object;
sub frobnitz {
$object->do_frobnitz;
};
print "Done";
... but also, creating and handing around other closures might create problems, especially if these closures are kept alive somewhere:
sub make_frobnitz {
my $frobnitz;
my $obj = $_[0]; # $object
$frobnitz = sub {
$obj->frobnitz;
};
return $frobnitz # $frobnitz might stay alive forever, keeping $ob
+j alive
};
Careful use of Scalar::Util::weaken or careful undeffing of $object within (one-shot) closures might help to locate and eliminate the problem.
If all else fails, defensive programming in the destructors also helps (taken from MozRepl::RemoteObject, which has this problem):
sub DESTROY {
my $self = shift;
local $@;
...
my $id = $self->__id();
return unless $self->__id();
my $release_action;
if ($release_action = ($self->__release_action || '')) {
...
};
if ($self->bridge) { # not always there during global destruction
my $rn = $self->bridge->name;
if ($rn) { # not always there during global destruction
...
} else {
# warn "Repl '$rn' has gone away already";
};
1
} else {
if ($MozRepl::RemoteObject::WARN_ON_LEAKS) {
warn "Can't release JS part of object $self / $id ($releas
+e_action)";
};
};
}
| [reply]
[d/l]
[select] |
Re: sub DESTROY: Strange ordering of object destruction
by moritz (Cardinal) on Mar 03, 2011 at 09:18 UTC
|
Some things to try:
- Try it on a different Perl version (App::perlbrew to the rescue)
- Compare reference counts with Devel::Peek before either object is destroyed
- Print out the references of A and B. Maybe the A you're seeing that should be holding a reference to B is really a near-duplicate of the object you think you are looking at?
- Try to isolate the code that shows the behavior, and show it here. (It doesn't need to be simplified actually)
- See if there is "nonlinear" control flow in program somewhere that might change things (signal handlers come to mind, threads or coroutines)
| [reply] |
Re: sub DESTROY: Strange ordering of object destruction
by ikegami (Patriarch) on Mar 03, 2011 at 09:51 UTC
|
This happens both on global destruction (exit) and controlled explicit destruction, i.e. undef $A.
Saying you used undef $A doesn't tell us anything about when the referenced object was destroyed.
You should only get the stated behaviour during global destruction, and I find it hard to believe it has occurred at any other time.
Without a demonstration, we can't really give anything but coulds and shoulds.
| [reply]
[d/l] |
|
|
END {
warn "Global Destruction starting";
}
Sure enough, the destruction is happening during global destruction, despite the undef $A much earlier. As clarified in the CB, during global destruction objects are destroyed in any order, so one should never assume that just because A holds a reference to B that A a will be destroyed before B.
To be honest, this still surprises me. Once I called undef $A there were no more variables storing a reference to A. i.e. the code was essentially nothing more than
sub {
my $a=SomeClass->new($arg1, $arg2, ...);
undef $a;
exit();
}
Wouldn't that undef result in the reference count for whatever was created by SomeClass->new going down to 0? If not, why not? There are no circular refrences in SomeClass - it stores only a reference to B and some pure scalars (strings, numbers). B is a third party class that has no knowledge of A and no way to pass in a reference to A, so a circular reference A->B->A isn't possible.
If yes, in the CB you said that destruction happens as soon as the reference count goes to 0. However, destruction is still happening only after the END block executes. Why?
In any case, thanks, again, for the END block suggestion in the CB.
Update: added comment about circular references. | [reply]
[d/l]
[select] |
|
|
Maybe the object itself keeps itself alive? Maybe it has child objects that keep the parent alive too?
My guess is that your object SomeClass will never get released until global destruction.
My advice is to rip out code from SomeClass until it starts getting released at the appropriate time, and then slowly putting the code (and the SomeClass-internal calls to the code) back in until the leak reappears.
| [reply]
[d/l] |
|
|
|
|
| [reply] |
Re: sub DESTROY: Strange ordering of object destruction
by anonymized user 468275 (Curate) on Mar 03, 2011 at 09:52 UTC
|
The bit about going out of scope bothers me. When something goes out of scope, it isn't always destroyed and may yet come back into scope later with its values conserved. Imagine you are asked to prove your assertion that the unwarranted destruction is taking place - what evidence would you present to support this (show the code that prints or examines the reference) and how would you argue that it leads to your conclusion?
| [reply] |
Re: sub DESTROY: Strange ordering of object destruction
by Anonymous Monk on Mar 03, 2011 at 10:22 UTC
|
Maybe buffering is confusing your debugging efforts?
#!/usr/bin/perl --
use strict;
use warnings;
Main(@ARGV);
warn "\nCommencing GLOBAL DESTRUCTION PHASE";
exit(0);
sub Main {
{
my $mya = MyA->new;
my $myb = MyB->new($mya);
my $myc = MyC->new($mya);
undef $mya;
}
warn "\nstill in Main";
my $myc;
{
my $mya = MyA->new;
my $myb = MyB->new($mya);
$myc = MyC->new($mya);
undef $mya;
}
warn "\nReturning from Main";
}
BEGIN {
package MyParent;
sub new { my($c)=shift; bless {@_}, $c }
sub DESTROY { warn "DESTROY @_ "; }
$INC{'MyParent.pm'}=__FILE__;
}
BEGIN {
package MyA;
use parent qw' MyParent ';
$INC{'MyA.pm'}=__FILE__;
}
BEGIN {
package MyB;
use NEXT;
use parent qw' MyParent ';
sub new { shift->SUPER::new( MyA => @_ ); }
sub DESTROY {
warn "BREAK LEAK @_ ", delete $_[0]->{MyA};
$_[0]->NEXT::DESTROY;
}
$INC{'MyB.pm'}=__FILE__;
}
BEGIN {
package MyC;
use parent qw' MyParent ';
sub new { shift->SUPER::new( MyA => @_ ); }
sub DESTROY {
warn "LEAK @_ ", $_[0]->{MyA};
$_[0]->NEXT::DESTROY;
}
$INC{'MyC.pm'}=__FILE__;
}
__END__
LEAK MyC=HASH(0xa47114) MyA=HASH(0x3f8d4c) at - line 56.
DESTROY MyC=HASH(0xa47114) at - line 31.
BREAK LEAK MyB=HASH(0x9c9864) MyA=HASH(0x3f8d4c) at - line 45.
DESTROY MyA=HASH(0x3f8d4c) at - line 31.
DESTROY MyB=HASH(0x9c9864) at - line 31.
still in Main at - line 17.
BREAK LEAK MyB=HASH(0xa47174) MyA=HASH(0x9c9864) at - line 45.
DESTROY MyB=HASH(0xa47174) at - line 31.
Returning from Main at - line 25.
LEAK MyC=HASH(0xa47124) MyA=HASH(0x9c9864) at - line 56.
DESTROY MyC=HASH(0xa47124) at - line 31.
DESTROY MyA=HASH(0x9c9864) at - line 31.
Commencing GLOBAL DESTRUCTION PHASE at - line 7.
| [reply]
[d/l] |
|
|
If you add this before Main(@ARGV)
our( $GlobalMya , $GlobalMyb, $GlobalMyc );
$GlobalMya = MyA->new;
$GlobalMyb = MyB->new($GlobalMya);
$GlobalMyc = MyC->new($GlobalMya);
you should a difference like this
| Commencing GLOBAL DESTRUCTION PHASE at - line 8.
|
|
| Use of uninitialized value in warn at - line 51 during global destruction.
|
|
| BREAK LEAK MyB=HASH(0xa4762c) at - line 51 during global destruction.
| LEAK MyC=HASH(0xa4758c) MyA=HASH(0x3f8d4c) at - line 62.
|
| DESTROY MyB=HASH(0xa4762c) at - line 37 during global destruction.
| DESTROY MyC=HASH(0xa4758c) at - line 37.
|
| DESTROY MyA=HASH(0xa475ec) at - line 37 during global destruction.
| BREAK LEAK MyB=HASH(0x9c9694) MyA=HASH(0x3f8d4c) at - line 51.
|
| Use of uninitialized value in warn at - line 62 during global destruction.
|
|
| LEAK MyC=HASH(0xa4765c) at - line 62 during global destruction.
| DESTROY MyA=HASH(0x3f8d4c) at - line 37.
|
| DESTROY MyC=HASH(0xa4765c) at - line 37 during global destruction.
| DESTROY MyB=HASH(0x9c9694) at - line 37. |
I've tried with 5.8.9 and 5.12.2, only the global destruction order varies between the two versions
| [reply]
[d/l] |
Re: sub DESTROY: Strange ordering of object destruction
by JavaFan (Canon) on Mar 03, 2011 at 11:29 UTC
|
Try reducing your program to a minimal, stand alone, program that exhibits your problem. Without code, all we can do is guess.
And my guess is that in the process of trying to replicate the issue in a small program, you'll find the cause. | [reply] |
|
|
Code reduction is certainly the gold standard but in this case the class is very complicated and doing it right would take some time. I posted because I was looking for background knowledge or experience with DESTROY that might explain the situation and allow me to avoid that, or at least, ensure that I was checking for the right things when I parred away code.
In this case, asking first proved quite fruitful. First, confirmed my suspicion that objects are deleted in indeterminant order during global destruction. Secondly, it caused me to question my assertion that destruction was in fact happening before the global destruction phase.
Had I reduced the code first, I would have been using inadequate tools to determine the order of destruction, and so my reduced code, for all the work it would have caused, would not have provided any more information than the more complex code.
Had the consensus been that there was likely a bug, I most certainly would have gone the code reduction route, if only to produce a repeatable test case. A bug report without a test case is the pits.
| [reply] |
|
|
