It is not clear however why you don't simply put each perl build in a separate hierarchy

I used to do that, but the repeated duplication of hunks of perl didn't fit well with my frugal nature and my sense of orderliness.
Perl already offered the option to take this alternative approach, so I thought "why not".
I would have gone ahead with the change even if I had been aware of this issue - and I'm still happy that I did go ahead with it.

If it does matter, that suggests either that there are tests missing for Module::B, or that there is code in one module that should be in the other

It's quite possible for bugs to exist for just one architecture - maybe because a pre-requisite is at a different version, or because a need specific to one architecture has been overlooked.
I just want to do what I can to ensure that any such bugs affecting my module are caught.

Cheers,
Rob