If you have a circle, then any value in that circle will repeat infinitely often. So at depths 2, 4, 8, and so on you switch to looking for the current element at deeper depths. When you back up, you stop looking for a match. At any point in time you only need to remember your current depth and possibly the one element that you're looking for duplicates of.

This is guaranteed to find any circular references. It just won't always find them promptly, and so in a total size count you might count elements more than once.