It depends. If your module for instance is an API to a certain site, than it only makes sense to actually connecting to the site. Also, if the purpose of a module is to connect to some remote server, then by all means, a module should have tests for that. It's no different than say, a DBD driver containing tests that actually connect to a database.

This may require some setting up to be done by person running the test suite. Just be sure it's documented. And die with a sensible error message if no connection can be made.