Yes, if you're testing core functionality. Those tests should be run on every machine. You're going to be the one answering the emails if someone installs something on their machine and there are no tests to catch the machine-specific failure. If a user doesn't want to install those Test::* modules, then they don't have to run make test and they can do it manually.

Where I make exceptions is in t/pod.t and t/pod-coverage.t, because they don't affect the functionality of the code, and won't differ in the results on other platforms than my own. See http://use.perl.org/~petdance/journal/19950 for examples.

Also, anyone reading this may be interested in some of the testing modules listed at http://qa.perl.org/test-modules.html.