I tought there was a function in Scalar::Util that returns the refcount of an object, but I couldn't find it. But also in Scalar::Util, you find the weaken function, which lets you pass around weak references. From your description, I think you have a "master" object that holds all "relevant" children, but over time, some of the children become irrelevant yet are kept alive by some references. If you make your master hold strong (normal) references but pass out only weak references, your children will be removed if the master removes them from the list.

Such an approach will have different problems, because weak references can turn into strong references again if you copy them, unfortunately. Maybe another approach would be to (always) ask the master whether a child is still relevant or not ...