Although voting affects the owner of the node too, don't forget that voting is mainly setting a value for the node itself.

Please do not get upset about the kind of votes you get on your nodes. Most of the time it is directed at the node itself, and not at you personally.