Thoughts for the problem which may help solving the issue:

Any ref which has to be tracked has to exist at object-creation-time

Any newly added ref will be added through a method - and will end up in {changed} anyway

I could walk down the tree at creation-time of the object and tie() all ref's to a helper package but this doesn't feel like a good solution. The same for creating a deep copy at creation time and deep-compare it to {data} just before the changes are flushed to storage.