Try it on a different Perl version (App::perlbrew to the rescue)
Compare reference counts with Devel::Peek before either object is destroyed
Print out the references of A and B. Maybe the A you're seeing that should be holding a reference to B is really a near-duplicate of the object you think you are looking at?
Try to isolate the code that shows the behavior, and show it here. (It doesn't need to be simplified actually)
See if there is "nonlinear" control flow in program somewhere that might change things (signal handlers come to mind, threads or coroutines)