The precedence for || is pretty high. Your unlink would give a list context for glob(), but || gives it a scalar context instead. What you wrote is parsed as:

unlink ( glob(...) || print "..." );
[download]

This might show what's going on a little more:

sub print_context {
    my $wa = wantarray;
    print $wa ? 'list' : defined $wa ? 'scalar' : 'void';
    print "\n";
    return $wa ? (1, 2) : 'x';
}

print "with || :\n";
print print_context() || print "error";
print "\n";

print "with or :\n";
print print_context() or print "error";
print "\n";
__END__
with || :
scalar
x
with or :
list
12
[download]

Anyway, the short answer is to use or instead of ||, or use parentheses around the argument to unlink. These should work:

unlink( glob(...) ) || print "...";

unlink glob(...) or print "...";
[download]