I wonder if I'd even notice if someone downvotes every node I ever wrote.

we do have lists sorted by reps, not only best nodes but also the order of replies are effected.

and authors will start to question their motivation to participate if they repeatedly experience immediate downvotes as first reaction.