This month I voted 199 times, this year only 72 people have voted as much or more so yes you're right we like the idea but not the action.
The main problem is that the pool of people who are engaged with Server Fault is quite small and diminishing. The vast majority of people have no engagement beyond getting their question answered, there are ~1500 people whose total reputation is >200 out of 139000 people who have used the site.
There is no point in bailing any more - too few people and too much crap to vote on.
Even though I try not to play too hard and actively avoid /review I still manage to be 3rd highest voter this year and 6th this month.