As I said, I have tests for my own application that exercise the parts of the modules they use that I care about.

The majority of CPAN modules don't fall into this category and failing 'make test's almost always mean that the modules aren't going to work as expected and really should just as well be considered build failures.

My experience has been just the opposite -- failing tests on CPAN modules that I've installed and used successfully before hardly ever indicate anything useful. They usually indicate a non-portable assumption in the tests themselves. I run the tests when I'm trying new modules, or when something doesn't work for me, but I don't run them to verify that a configuration I've used before still works.