I can't quite figure out if you want the elements in the array but not the hash, or in the hash but not the array. But here's how to get both.

my @not_in_hash = grep {!exists $hash{$_}} @all_ids;

my %only_in_hash = %hash; #make a copy first
delete @only_in_hash{@all_ids};
[download]