It is very rarely that I downvote a reply, rather my standard practice is to neglect to ++ it if it is a repeat or slightly incorrect. When I began to vote, I found
How should I spend my votes? -- General Voting Guidelines very helpful, and I tend to find that most replies to my questions deserve a ++. Only those that are really cruel to me do I downvote.
Of course, I agree that it does seem slightly awkward downvoting those who tried to help you, so I really don't. Nodes that are way OT, trollish, or rude tend to be the only ones I do it to.