It's not a perfect system and it is something that worries me about the bounty system. Yes, the bounty can be awarded to someone automatically that has not answered your question, but has somehow tickled the community just enough to gain upvotes.
I think that it is not known by enough people that the highest upvoted question will have the bounty awarded to it and therefore upvotes are not always given as carefully as they should be. My memory recalls some bounty answers that made me stop and think "Why on earth would someone upvote that? That's not a cogent or contextual answer..." That problem is certainly not limited to the bounty system, as plenty of answers across ServerFault have made me stop and wonder that. However, when posterity is at stake via automatic awarding, it seems to be a bigger issue.
Solutions wanted.
EDIT: Upping the required amount of upvotes for a bounty to be half-awarded would be good. Five vote minimum?