There may be better solutions depending on just what you need to do, but it looks to me like you could set a flag on the object to say that it is now a backend object. Somewhat clunky, but maybe a low effort way to fix the problem.