I'm guessing the Chris Dolan link you want is Private Regression Tests.

Chris recommends creating test files such as 00_local_foo.t and adding \d+_local_ to MANIFEST.SKIP.

I'd probably do something as you say—set an environment variable to tell Test::More whether to skip_all or not. If I were feeling too clever for my own good, I'd check against stuff like the local hostname, username, or whatever.