Other than cyclic references (which I don't think could exist in my code), do you know what could cause this to happen?

Well, circular references do not have to be direct. There are lots of subtle ways for creating them. That makes finding them very complex at times.

I don't know about any other reason for perl to keep unused objects around.