up vote 3 down vote favorite

I have been noticing quite a number of "test" questions and answers being thrown into the mix. When did this start and what is being done with the information obtained?

share
I heard about it on the podcast over a month ago. Though at the time I was supposedly pretty rare. I suspect they may have increased the frequency. – Zoredache Jan 30 at 19:05

1 Answer

up vote 7 down vote accepted

There have been some complaints elsewhere (er, SO) that certain Reviewers are just clicking buttons as fast as they can just to get badges or to whittle down the queues. Because of this, they started throwing speed-bumps into the mix in the form of already-reviewed questions (and mod-hammered ones if I'm remembering right) to cross-check Reviewers against community standards.

If you get enough Fails, you're locked out of Review for a while

The details are found here: Why was I told, "Congratulations, this was only a test" when reviewing?

They've only recently added these to SF and SU, so they're definitely still tweaking the system.

share
1  
They need to tweak that a lot. Having to experience this really annoys me. – SvW Jan 30 at 20:09
1  
Awesome! This is a sorely needed feature. – WesleyDavid Jan 30 at 20:21
1  
The problem I forsee is that I disagreed with a couple of the "right answers"....also this may cause the number of reviewers to drop off. Each user has his perspective and that is what makes this system work, testing generates more work and some frustration. – mdpc Jan 30 at 21:16
1  
@mdpc That's kind of the point, actually. If you're too far off of the standards everyone else is reviewing against, you do need to be told that. – sysadmin1138♦ Jan 30 at 21:24
what about diversity in opinion? – mdpc Jan 30 at 21:49
@mdpc That's what the review queues are all about. The items thrown up already have been voted on by diverse people (or one super-user). The items you vote on that aren't tests supply your view to the overall body of reviewed-posts. This 'audit' system is designed to herd people closer together. – sysadmin1138♦ Jan 30 at 21:56
1  
Can't we just bin the review system and carry on as we did before it came? It is full of wrong incentives and favors herding behavior. – syneticon-dj Jan 31 at 21:10
It's a powerful training tool - I've been quiet for a while, thought I'd start up again by doing some reviews and can now see that my "view" isn't up to par. Not going to review or edit again until I've had some fresh answers graded. Not sure if it's a good or a bad thing, but it certainly made me think more carefully about the purpose of reviewing. – Helvick Feb 10 at 12:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged