I'm kind of with choedebeck here -- I'm unsure as to where you'd be unsure.

A passing test means "I expect the bit of functionality I just tested to work". A failing test means "I expect the bit of functionality I just tested to not work". A skipped test means "I couldn't tell if this bit of functionality will work for you or not".

On the other hand, I interpret the env vars bit the other way. If the env var is needed for the /test/, but not the /functionality being tested/, then you should skip, since you can't say anything about the functionality. For example, if you have a module that accesses an online service, and the normal way of giving it an account is to pass the username and password to new, you need to give some way of letting the tests get a username and password. Obviously, if you don't have a username and password, you can't test much in this hypothetical module, and thus won't know if it'd work or not if they did pass something to new. If they didn't, then they aren't using your module correctly, and there's little way to test against misuse.

There are only a few cases I can think of where a skipped test would be appropriate. Operating system specific code comes to mind. As does optional functionality that is dependant on packages the user doesn't have installed. Other than that I don't know why you would skip.

There might be functionality only available on high enough versions of Perl - if the installed version is lower, you may want to skip tests. And some functionality (or tests) may depend on compilation settings. There's no point in testing if the module runs under threads if the installed version of Perl doesn't have threads enabled. You also might want to skip tests if the installed version has 32bit integers instead of 64 bits.
Perl --((8:>*

"Todo" tests might be of interest to you. from perldoc Test::More

    TODO: {
        local $TODO = "URI::Geller not finished";

        my $card = "Eight of clubs";
        is( URI::Geller->your_card, $card, 'Is THIS your card?' );
        my $spoon;
        URI::Geller->bend_spoon;
        is( $spoon, 'bent',    "Spoon bending, that's original" );
    }
[download]

With a todo block, the tests inside are expected to fail. Test::More will run the tests normally, but print out special flags indicating they are "todo". Test::Harness will interpret failures as being ok. Should anything succeed, it will report it as an unexpected success. You then know the thing you had todo is done and can remove the TODO flag.

There's a distinction between TODO tests and SKIP tests, in that TODO tests are actually run, but SKIP tests are not.

Conditional skips are used for tests that are a) inappropriate for the platform, b) inappropriate for the user's responses to questions in Makefile.PL or Build.PL c) an optional module is not installed. Sometimes you might skip the whole .t file with plan skip_all => reason.

TODO tests are for "scaffolding" - code that hasn't been written yet, and tests are expected to fail. TODO tests that fail more severely, such as dieing, can be avoided with todo_skip (see the docs).

Hope this helps

My feeling is that the installer should notify the user of the missing dependencies and give them a chance to bail out / install the dependencies instead of just skipping the tests. You should probably use ExtUtils::MakeMaker's prompt() function for this.

Example snippet from a Makefile.PL:

my $play_audio = 1;
if (! eval { require Audio::Play; } ) {
    $play_audio = prompt(
"No Audio::Play module was found on your system. Without it, 
we can't do audio output. If you want to test audio output,
answer yes to the following question. Audio::Play will then 
be added to the required module list. This means it will be
installed automatically if you're using the CPAN / CPANPLUS
modules.

Do you want to test audio output (requires Audio::Play)? y/n","n") =~ 
+/^y/i;
}


WriteMakefile(
    'NAME'      => 'Audio::LADSPA',
    'VERSION_FROM'  => 'LADSPA.pm', # finds $VERSION
    'PREREQ_PM'     => {
    'Test::More' => 0,
    ( $play_audio ? ('Audio::Play' => 1.000, 'Audio::Data' => 1.000) :
+ ()),
    'Graph' => 0.5,         # NEW interface for graph
    'Scalar::Util' => 0,
    'Data::Uniqid' => 0,
    }, # e.g., Module::Name => 1.i

    ($] >= 5.005 ?    ## Add these new keywords supported since 5.005
       (AUTHOR     => 'Joost Diepenmaat <jdiepen AT cpan.org>') : ()),
    'EXE_FILES'     => ['eg/pluginfo'],
    'OPTIMIZE'      => '-O',
)
[download]

The reason for this particular skip is that Audio::Play seems to have some issues installing on certain systems, and it's not strictly nessecary for the module in question. The test/*.t files will skip if Audio::Play can't be loaded.