The CPAN testers will automagically test your module with various configurations. Unfortunately, the later CPANPLUS (as of v0.0050) all broke cpan smoke testing, but luckily, CPAN::YACSmoke picked up the baton.

You will be notified of failures, maybe together with a detailed report of the failure. You can also view the failures via your CPAN page, via the "CPAN Testers" link.

Update: Put in correct link to CPAN Testers