9

What tools or metrics do we have to measure the health of the ServerFault community? I mean that are available to non-moderators.

I've read lots of debates here over the years about how healthy the community is. I'm not trying to reopen those. But just now, I was looking at the Questions feed on the home page, and I noticed that of the questions on the first page:

  • 20 were from authors with reputation 1
  • 15 were from authors with reputation 2-300
  • 3 were from authors with reputation 301 or higher

It's great that we have junior sysadmins who come to learn, but this seems like an unhealthy balance. But it's just anecdotal.

So what metrics can we go and look at that try to answer the health question?

8
  • 3
    The tool is of course: data.stackexchange.com/serverfault - the question is of course what to query to measure "health"
    – Bob
    Commented Mar 19, 2021 at 15:32
  • 3
    How many were answered ? What was the quality of the answers ? What was the reputation of the answerers?
    – user620588
    Commented Mar 21, 2021 at 9:46
  • 1
    data.stackexchange.com/serverfault/query/303570/…
    – user620588
    Commented Mar 21, 2021 at 9:49
  • 1
    Most of the so-called 'junior sysadmins' probably wouldn't consider themselves to be so. check out their profiles and see where they have most reputation. It will most likely be SO which makes them developers. When I did this a few years ago >60% of questions were asked by people with more rep elsewhere in SE. Te remainder were either amateurs or it could not be determined.
    – user9517
    Commented Apr 8, 2021 at 15:08
  • @user9517 A trend I observe is that karma doesn't always tell the whole story. For example, I have the highest karma over at AskUbuntu, but that is only because I hit the jackpot on one question. The majority of my activity has been elsewhere, for example here in SF I have >1400 reviews as I used to spend a lot of time in the review queues I have access to (though I'm not a professional so mostly I was just making sure people were following the site's code conduct, revising questions with poor formatting, encouraging users to ask better questions, etc.).
    – Paul
    Commented Apr 14, 2021 at 12:28
  • @user9517 BTW, I'm not disagreeing, just commenting there are yet other factors.
    – Paul
    Commented Apr 14, 2021 at 12:29
  • 1
    @Paul Statistical noise.
    – user9517
    Commented Apr 16, 2021 at 8:27
  • @user9517 You are correct about SO in their inability to help their own users configure a server to support the dev work and instead dumping their questions over here. I had never noticed it before and, yeah, it probably is ~60% of the questions asked are users with substantially higher karma on SO. It also appears they aren't interested in reciprocating when I just ask a simple question requesting help for reading code in a plugin they promptly close the question because apparently anything to do with a server must be dealing with configuring a server (ultimately not the reason for asking).
    – Paul
    Commented Apr 16, 2021 at 17:16

2 Answers 2

Reset to default
11

Question and Answer count

Thanks @user620588. The number of questions that are asked versus the number of questions are answered may indeed make a metric to determine the site health.

Graph from https://data.stackexchange.com/serverfault/query/303570/answer-rate-over-time#graph

That graph shows a steady decline in the absolute number of questions asked and answered on ServerFault.

So by that metric the site is in decline and has been for years.

2020 saw a trend reversal, but only in the number of questions asked rising sharply.
(Maybe because people working from home due to restrictions get less mentoring and/or are less inclined to "bother" their colleagues to ask a question and turn to online resources instead?)

The absolute number of answered questions in 2020 didn't show any increase at all, but rather continued the downward trend. So percentage-wise much more questions remain unanswered now. That aggravates the appearance of decline of the site.

Those numbers suggest more sand and fewer pearls...
https://stackoverflow.blog/2011/06/13/optimizing-for-pearls-not-sand/

Voting

Since you need to be registered user to be able to vote the number of votes cast might be another metric to show “community” and participation :

graph from https://data.stackexchange.com/serverfault/query/1375382/up-accept-and-down-votes-over-time

That metric shows a different pattern, at first glance despite some fluctuations the number of votes in the largest categories appears to remain quite steady.

That suggests that the number of additional questions posted in 2020 did not result in an increase of the actively participating community, but it doesn't suggest a decline either.

8
  • 3
    One thing we don't know is how many people ask a question of $favoriteSearchEngine, get directed to SF, find the answer to their problem and move on (having viewed all of the relevant adverts etc.) . The widening gap between questions asked and answered is not good. I wonder what that would be like if Michael Hampton stopped playing the game. Maybe the folks at Octopus deploy can tell us whether their money was well spent in sponsoring the site ?
    – user9517
    Commented Mar 22, 2021 at 16:53
  • 2
    @user9517 I often come here to search and usually find answers to my questions, so I don't even come in through a search engine. I do often leave upvotes, though I find I'm more generous than others when it comes to that. Also, it gives me great joy to see Michael Hampton returning, but he does often answer questions in a comment, which skews the above graph. Further, search engine answer boxes often include the relevant parts of an answer in the search result, meaning the user wouldn't even have clicked on the search result (or any, as is now a majority trend of "zero-clicking").
    – Paul
    Commented Mar 23, 2021 at 16:20
  • 1
    How quickly bad/spam/duplicate questions get closed is also something to consider. My impression is that all categories hang around far too long as there just aren't enough people who care any more.
    – user9517
    Commented Apr 8, 2021 at 15:10
  • @user9517 Is this related to the close votes review queue? There is a lot more in the review queues overall as compared to times past. I used to spend a lot of time in the review queues I am granted access to, but haven't in some time because reasons, and I wonder if attrition of reviewers is higher than previously.
    – Paul
    Commented Apr 9, 2021 at 13:51
  • 1
    @Paul It's not related to the review mines but the length of them is a consequence of the lack of people who play that part of the game. Thel ack of reviewers is a signal that things are less then good,
    – user9517
    Commented Apr 9, 2021 at 19:32
  • @user9517 I have wondered if it is related to up-vote stinginess. If a new user comes in and asks questions only (as they are learning), my casual observation is that the questions tend not to get up-votes. Even when a question generates an answer that is accepted, and that answer gets up-votes, the question often does not. I do realize accepting an answer yields karma reward, but it's really slow going for new users. This means it takes longer to build up karma to gain access to things like review queues, and likely many never bother as it may appear unachievable.
    – Paul
    Commented Apr 13, 2021 at 17:02
  • 1
    @Paul See my last comment on the question above. SF is SO's Hell Desk. Most of the questions are asked by people who just want to get on with hacking their code. Using google and thinking is too time consuming for them and their primary goal.
    – user9517
    Commented Apr 14, 2021 at 8:08
  • @user9517 I'm not trying to turn this into an SO-bashing session, but I just posted a question over there that would be on-topic here at SF, however I thought it would be there because it's basically my inability to read through a spamassassin plugin. It was closed and you can see the reason: stackoverflow.com/questions/67094667/…
    – Paul
    Commented Apr 15, 2021 at 1:07
2

YOU CAN'T

Dead things have no useful health measure.

When was the last time the Close votes queue was empty? Or, for that matter, when was the last time someone reviewed anything in the Close votes queue?

Several times this week I've gone down the questions sorted 'Newest' and 'Active' and what a disappointment. I used to find at least one or two juicy questions (and with great answers!) and would regularly learn stuffs that way. Now just garbage and no or poor answers.

I am now in agreement with Iain that SO has destroyed this site by flooding it with dev-support questions. Since the devs are highly important thus cannot be bothered to stoop to the lowly level of contributing back with answers here for other SO devs, the site just continues in decay.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .