How about using the hash for the banned keys? Something along these UNTESTED lines:

sub subtract {
   my ($whole_ref, $banned_ref) = @_;
   my %is_banned = map { $_ => 1 } @$banned_ref;
   return grep { ! $is_banned{$_} } @$whole_ref;
}

my @surviving = subtract([qw(fetch forward user smtp)], [qw(user smtp)
+]);
[download]

This will also preserve the ordering in the original "whole" array. Note that array references are passed to the subtract() sub, in order to be able to pass two arrays (the first with the "whole" list and the other with the "banned" one).