If you find the prospect of losing a few XP distasteful, then you will need to find at least a few more-deserving nodes so that you can up-vote them and get your "voting average" back above zero for a while. You also might want to consider not being so quick to down-vote nodes (considering how distasteful you find losing XP to be - remember, Monks lose XP when their nodes are down-voted). Down-voting nodes for relatively minor reasons can often be a bit disruptive to our community.