On thing the perl-qa list settled on a while back is to put release-specific tests (or, if you like mnemonics, eXtra Tests) in the xt/ directory.

This effectively moves such tests out of the way of standard builds on end-user machines. Future versions of ExtUtils::MakeMaker and Module::Build will offer a way of running the extra tests in xt/ with a minimum of fuss. (If in fact they haven't already done so; I haven't looked recently).