I have such a test in my Perl distribution template.

The test is a (not so) simple glue of Perl::PrereqScanner::Lite to find all used modules and some other stuff to remove known modules, and it (re)runs the test suite with each module blocked in turn.

This test basically works, except for the cases where I dynamically use different modules based (for example) on the operating system or other things that the test suite will not pick up on, because it (for exmample) only runs on Linux while a module is dynamically required on Windows.