my $fh;
BEGIN {
    open $fh, '>>', '/var/foo.log' or die;
}
use My::Module $fh;
[download]

could be written as

use My::Module do {
    open my $fh, '>>', '/var/foo.log' or die;
    $fh
};
[download]

But it's still rather icky.

As for validating the handle, I suggest you don't. You'll get an error when you try to use the bad argument as a file handle, and that should be good enough here.