If a question seems interesting to me, or potentially practical, I vote it up and may star it. If it does not (by being outside of my field of interest/competention) I just go my way ignoring it. If a question seems inappropriate/offensive I flag it. If a question seems to be duplicate or better corresponding to another Stackexchange site - I vote to close/move it.
So what's the case when I am logically meant to downvote?