Some things to try:

• Try it on a different Perl version (App::perlbrew to the rescue)
• Compare reference counts with Devel::Peek before either object is destroyed
• Print out the references of A and B. Maybe the A you're seeing that should be holding a reference to B is really a near-duplicate of the object you think you are looking at?
• Try to isolate the code that shows the behavior, and show it here. (It doesn't need to be simplified actually)
• See if there is "nonlinear" control flow in program somewhere that might change things (signal handlers come to mind, threads or coroutines)