Thanks, chromatic and Joost! I ended up doing this:

my $have_test_warn;

BEGIN {
    eval {
        require Test::Warn;
        Test::Warn->import();
        $have_test_warn = 1;
    };
}

SKIP: {

    skip 'Test::Warn not installed', 4 unless $have_test_warn;

    ....
}
[download]

I didn't assign the value to $have_test_warn in the declaration. If I did, it is executed before it gets to the SKIP block, and the value is 0 again.