My general objection to this idea is not that it cannot be done, but rather that it will introduce a “bolt out of the blue” effect into the system. An apparently-unrelated bit of code over there has a very draconian effect over here, and I might not know to look “there” to diagnose what appears to be a bug “here.” Bear in mind that the hardest thing about resolving any bug – by far – is finding it.

So, if you have a phalanx of references to something, to anything at all, then this might suggest a weakness in your overall design. It is better, in my opinion, for there to be one and only one “custodian of” any Thing of Interest. If other objects must maintain temporal references to it, consider instead having them “ask the custodian to find it.” Or, if that isn't practical, consider setting up a notification method in the various objects that are designed to have a reference to this Thing, so that when you need to destroy this Thing you can call this method and thereby cause each object to set its own reference to undef; to cause them to break their link to this Thing.

(n.b. Don't go perldocing for “notification method.” That's my colloquial term.)

What advantage does that have? Well, instead of doing something “to” them, from somewhere-else totally unexpected, you instead cause each object to do what needs to be done upon themselves. It explicitly documents the fact that this link can be broken, and it explicitly creates a highly-visible point in the code ... for this object ... when-and-where it actually happens.

Now, all of the code for “this object” is once again self-contained and self-explaining. When you look at the package which defines this object, you see all of the code that materially affects it. It's all right here.