I rarely upvote stories, unless they were either very interesting, or were interesting but didn't seem to have enough votes yet.

I upvote comments when they raise an interesting point that I hadn't though of before (even if I disagree) in an intelligent way. I rarely upvote just because I agree with something, and I never downvote out of disagreement.

I only downvote when the poster clearly has no idea what they are talking about it. (Or is just being pedantic and critical to no apparent benefit).