49

Today, 5th June, is the beginning of a strike by many StackExchange moderators over the recent policy announcement by StackExchange staff around handling AI and Large Language Model (LLM) generated content. You can read the strike announcement yourself at: https://openletter.mousetail.nl/ plus another detailed take at https://jlericson.com/2023/06/04/signing_on.html

For those who have not heard of this policy change—or don't want to click the link—the StackExchange staff announced that the use of AI-detection tooling to identify AI/LLM generated content for the purposes of deletion/bannning is now banned. Their justification for this move is two-fold:

  • After testing, they've found these tools have a quite high false positive rate. Possibly up to 30%.
  • These tools disproportionately affect people for whom English is a second language. AI/LLMs let them turn broken English into grammatical English, at the cost of now failing an AI/LLM detector.

The strikers argue that this cost is justified given the unique threat that AI/LLM represents to question and answer sites like ours.

  • We've already seen spammers use these tools to provide an actual-answer seeming post with a spam link in it.
  • The setup of the StackExchange sites is well suited to provide human qualilty-assurance for AI/LLM generated content, and that's not why we're here.
  • AI/LLM content can be created far faster than humans can audit them, even if that was a role we were interested in.
  • For sites with far more community than we've had in the last few years, a deluge of AI/LLM content will break that community.

The strikers will be ceasing moderation activities until this policy is reversed. Also of note, this includes halting the SmokeDetector anti-spam bot. Since late February, ServerFault has been under heavy attack by a group of pirate stream spammers. Most people haven't seen them because SmokeDetector spam-flags them out of existence within 5 minutes in most cases. With the SmokeDetector turned off, you will begin to see this traffic.

I will be respecting this picket line.

7
  • 17
    Why was this un-featured?
    – vidarlo
    Jun 5, 2023 at 10:09
  • 13
    @vidarlo The company doesn't want anything mentioning the strike featured. Not even a joke, we were told so in advance (but in secrecy, so a lot of mods have missed it. Even then, not really much harm in trying)
    – Zoe
    Jun 5, 2023 at 10:52
  • 12
    I would very much like an official answer to that effect. Cowardice and hiding doesn't really build trust currently.
    – vidarlo
    Jun 5, 2023 at 11:00
  • 4
    Indeed, but I doubt we'll get one. There's a strike for a reason
    – Zoe
    Jun 5, 2023 at 11:16
  • I agree. I also note that it's not consistently enforced; over on AU it's still two featured posts about the strike.
    – vidarlo
    Jun 5, 2023 at 11:28
  • 9
    Adding back the featured tag, as any strike action can have an impact on the site expectation, the user must be directed on the why. (I waited as the post was already in the hot meta post list, so the tag was less important)
    – yagmoth555 Mod
    Jun 6, 2023 at 20:52
  • 1
    As of 2023-08-02, a major round of negotiations that had been organized outside the Meta sites has concluded. This Meta.SE describes in detail what agreement has been accomplished at that table: Moderation strike: Results of negotiations
    – anx
    Aug 2, 2023 at 20:29

4 Answers 4

34

To expand on this:

announced that the use of AI-detection tooling to identify AI/LLM generated content for the purposes of deletion/bannning is now banned

I'll add two points:

  • At least on Stack Overflow, we've not been using any automated detector as a primary (or for most/all of us, any) factor in removing generated content. It's been plainly obvious they were unreliable since about week 2 of this crisis.
  • The edicts from staff in private go further than most publicly available statements. To follow the private rules, we are unable to remove AI content outside of the narrowest of circumstances. Effectively, we are not allowed to remove content on the basis of it being AI-generated regardless of detection method.
25

I think it's important to remember that not only those with diamonds moderate. We all do - through up and down-voting, through closing, flagging and so forth.

It's whats left of the community at stake.

I will not do any moderation actions, and have signed the letter.

22

I have not played the Serverfault game with any gusto for some time but my day job does land me here occasionally. When that happens I will often do some moderation.

I have signed the letter and will undertake no further moderation or curation of Serverfault.

11

Since this has gone live, StackExchange staff has provided rather more data to the Moderator community backing up their claims of AI/LLM-detection tools likely causing more harm. The details are not publicly released, so I'm not going to release it, but there is compelling evidence in there.

Most interesting to me, they used metrics only they have access to in order to heuristically determine who is likely using these tools to post content. Many of you know that the platform keeps 'hidden' versions in the background as you type. This dataset can be used to identify people who are pasting large blocks of text from other sources versus people like me who are typing a lot over the course of several minutes. This isn't an explicit ChatGPT-detector, but they shared a chart of answerers showing pasting behaviors over time.

That chart showed a large spike in the wake of ChatGPTs release, and a falling curve since. We hit parity with the pre-ChatGPT period around the first of May. This data is noisy, but when looking at periods of weeks is compelling that they've found a way to identify use of these tools. The key finding here is that AI/LLM usage isn't significant anymore.

They did another round of analysis, focusing on "people who post answers three or more times a week," which sounds like an internal "active users" metric they've been using for a while. Active users have been falling more than the long term trend as of about the first of December, and they're not sure why. Analysis hasn't given a clear answer, but one area that does seem to be affecting these numbers is the ban-rate of active users; a rate which has rocketed up in the wake of AI/LLM usage being deemed acceptable for sites to set policy about.

The ban-rate has not slacked at the same pace as detected AI/LLM usage, suggesting there is a high false positive rate. Supporting this, they revealed that they have an unprecedented rate of "unable to substantiate" results to appeals to moderator ban decisions. As in: they couldn't verify the reason for the ban.

This is data they should have posted with the earlier announcement. To their credit, this data was directly asked for and they've given it.


The data I was working from is now public: GPT on the platform: Data, actions, and outcomes

The community response to this is happening in real time.

4
  • 13
    "To their credit, this data was directly asked for and they've given it." After the community went on strike, yes. Jun 6, 2023 at 23:18
  • 6
    This is interesting, thanks! I realize you're walking a fine line here, but something I'd encourage pushing on is whether they've directly tracked the trajectories of users who were active (three months ago, 6, a year, 2 years, etc) - that is, tracking specific users, not the usual "x active users during y period" metric available on the usual charts. When I ran this in the past, I found a fairly stable fall-off in activity, albeit one that has changed over time: e.g., folks who joined in 2012 lasted a lot longer than folks who joined in 2016 and longer than folks who joined in 2011.
    – Shog9
    Jun 7, 2023 at 19:17
  • 6
    The reason I suggest this is that it would help separate effects caused by formerly-active users dropping off and those caused by fewer new users becoming active. It wouldn't necessarily point to a smoking gun here, but narrowing the (fairly vast) field of potential culprits is probably useful!
    – Shog9
    Jun 7, 2023 at 19:20
  • 2
    "In the wake of ChatGPT's release" - you mean ChatGPT-4 (aka Bing, aka Sidney)? Or are you referring to an earlier release?
    – Randall
    Jun 13, 2023 at 10:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .