How can I catch a failure of use?

You can do that by not catching it, by simply creating another more modules, so that  use MyStuff; just works

For example

package MyStuff;
use Module::Load qw/ load /;
my @mods  = qw/ 
MyStuff::Win32MostFeatures
MyStuff::Win32LessFeatures
MyStuff::LinuxOnly
/;
for my $mod( @mods ){
    if( load $mod ){
        push @ISA, $mod;
        last;
    }
}
scalar @ISA;
[download]

See Module::Pluggable, Module::Load ...