The only times I've really been downvoting is if someone is giving out completely incorrect information, like in a support thread or something, and confusing matters. It's not a personal judgement or anything, just trying to keep things clear for the person asking the question.
If I disagree with a comment, well no biggie. Sometimes it's worth discussing like adults and sometimes it's just a subjective opinion. If it's offensive, I'll report it and block the user.