I think modules and libraries are placed under a version directory under your prefix/lib/perl (by default) where as binaries like perl, perlbug, h2ph, h2xs etc by default will end up in prefix/bin so if you configure multiple perls with the same prefix your binary files get overwritten.