Depending on how persistently you hold on to objects, each object can have a weakref to the list of objects that it can use to remove itself from the list, and become unavailable once all references other than the list's go away.

If that's not good enough, make all references to objects other than the list's reference be to a delegate object that looks up the real object via the list with each access.