Jump to content

Wikipedia talk:WikiProject AI Cleanup

Page contents not supported in other languages.
Add topic
From Wikipedia, the free encyclopedia

Lumberjack essay about AI proofreading

[edit]

There was a good essay somewhere about the pitfalls of LLM-assisted proofreading somewhere. It featured an example about rewriting a short passage about an unnamed lumberjack. –LaundryPizza03 (dc̄) 16:07, 12 August 2026 (UTC)Reply

I don't recall that essay, but I do now want to rewrite The Lumberjack Song to "He's a lumberjack and he's AI"! Thryduulf (talk) 16:15, 12 August 2026 (UTC)Reply
Do it! Do it! M kuhner (talk) 18:21, 12 August 2026 (UTC)Reply
I remember this -- it was a table of possible edits to a sentence and subtle meaning changes introduced -- and remember contributing to it, but couldn't find it in my contributions, maybe it was deleted? Gnomingstuff (talk) 18:37, 12 August 2026 (UTC)Reply
If you can give me a hint of what you might have written in your edit summary, and a rough date range, I can look through your deleted contributions and restore it somewhere, like user space or draft space. ~Anachronist (who / me) (talk) 15:08, 13 August 2026 (UTC)Reply
I know for certain that I first encountered the page this year, probably May–July 2026. –LaundryPizza03 (dc̄) 17:37, 13 August 2026 (UTC)Reply
Your deleted contributions have about 2000 edits in the May-July range, more than 1000 of which are in May alone. If you have any clue that would narrow it down, let me know. ~Anachronist (who / me) (talk) 23:35, 13 August 2026 (UTC)Reply
@Gnomingstuff –LaundryPizza03 (dc̄) 14:20, 14 August 2026 (UTC)Reply
not sure why I was pinged to this, I have nothing more to contribute Gnomingstuff (talk) 14:48, 14 August 2026 (UTC)Reply
Because you surely remember something that can be used to identify the page.
@Anachronist: In this case, since Gnomingstuff claims to know nothing, I'd suggest doing a brute-force search. –LaundryPizza03 (dc̄) 15:28, 14 August 2026 (UTC)Reply
I tried searching for obvious strings like "lumberjack" or "LLM" or "proofreading" and others, and nothing came up in those 2000 edits. Do you remember if it was in someone's user space, or Wikipedia namespace? ~Anachronist (who / me) (talk) 17:34, 14 August 2026 (UTC)Reply
No. I'd lean toward checking Wikipedia namespace first. Keyword might be revision (as an uncountable noun) or proofreading. –LaundryPizza03 (dc̄) 17:38, 14 August 2026 (UTC)Reply
No luck. You hardly have any edits in the Wikipedia namespace (Wikipedia talk has much more). Here's what you have for the past 2 years:
  • 00:06, 13 August 2026 Wikipedia:Ahora en TODOS los Hipermercados Carrefour Bajamos los precios Requesting speedy deletion (CSD G3, CSD G11).
  • 00:05, 13 August 2026 Wikipedia:¡ENCONTRÁ EL REGALO IDEAL PARA EL DÍA DE LA NIÑEZ EN CARREFOUR! Requesting speedy deletion (CSD G3, CSD G11).
  • 00:03, 14 May 2026 Wikipedia:Sockpuppet investigations/Daisuke Roppunganai Requesting speedy deletion (CSD G6, CSD G7).
  • 00:02, 14 May 2026 −999 Wikipedia:Sockpuppet investigations/Daisuke Roppunganai Oops, misspelled Tag: Blanking
  • 00:01, 14 May 2026 N Wikipedia:Sockpuppet investigations/Daisuke Roppunganai Adding new report for Daisuke Roppunganai.
  • 04:31, 13 February 2026 Wikipedia:建立条目向导 Requesting speedy deletion (CSD G15).
  • 02:43, 5 January 2026 Wikipedia:WikiProject GuCe Editors/Balog elimination drives Requesting speedy deletion (CSD G1).
  • 19:22, 31 December 2025 Wikipedia:Articles for creation/Draft Requesting speedy deletion (CSD G2).
  • 07:49, 17 December 2025 Wikipedia:Cameron Jay Hauke Requesting speedy deletion (CSD G3).
  • 07:46, 17 December 2025 Wikipedia:Luzhou Weing Lei Bu Requesting speedy deletion (CSD G3).
Everything prior to that is 2022 and earlier. ~Anachronist (who / me) (talk) 17:46, 14 August 2026 (UTC)Reply
I have never edited the page you're looking for. I assumed, up to this point, that you were looking through Gnomingstuff's contribs, not mine. –LaundryPizza03 (dc̄) 17:48, 14 August 2026 (UTC)Reply
Unfortunately, Gnomingstuff has zero deleted contributions in the Wikipedia namespace, and the deleted contributions in user space all have an edit summary "u6 triage per policy discussion...". ~Anachronist (who / me) (talk) 17:56, 14 August 2026 (UTC)Reply
Is it possible that Gnomingstuff edited the page further back? It is also likely that if it was in Wikipedia namespace, there would be a revision to Wikipedia:Artificial_intelligence_resources that removed a link to the deleted entry. –LaundryPizza03 (dc̄) 18:05, 14 August 2026 (UTC)Reply
Okay, not that page, but possibly a different page in Category:Wikipedia and artificial intelligence or the WP:AINB archives linked to it. –LaundryPizza03 (dc̄) 18:06, 14 August 2026 (UTC)Reply
The lists on category pages are auto-populated without any changes to the revision history.
Gnomingstuff has zero deleted edits in the Wikipedia namespace for all time, not just this year. ~Anachronist (who / me) (talk) 20:41, 14 August 2026 (UTC)Reply
That second remark referred to the page histories of other pages in Category:Wikipedia and artificial intelligence, not the category itself. Maybe someone else familiar with this page can chime in. –LaundryPizza03 (dc̄) 00:01, 15 August 2026 (UTC)Reply
Is it okay if I ask WP:AN for input? If it was deleted, the deleting admin might know where it was. –LaundryPizza03 (dc̄) 06:14, 6 September 2026 (UTC)Reply
It wouldn't hurt. I'm interested too. Admins can search for deleted titles, but would need to know some words in the title and the name space. ~Anachronist (who / me) (talk) 14:49, 6 September 2026 (UTC)Reply
You don't need to be an admin to be able to do that. See Wikipedia:Database queries along with information about the archive table on MediaWiki.org. The only problem is that you can't access the texts and edit summaries of deleted edits that way. You don't even need to be an admin to find out about a user's deleted contributions, though it's way more awkward that way; see Wikipedia:Viewing and restoring deleted pages § Viewing metadata of deleted revisions. My query knowledge is relatively weak; the best place to request them is at Wikipedia:Request a query. I honestly wonder if the page was just overwritten rather than being outright deleted; deletions of userspace essays like that are rare but deletions of Wikipedia namespace pages are even rarer. Yes, there is deletion by author request, but does anyone know someone here in the AI space who got their pages deleted like that? Graham87 (talk) 04:17, 7 September 2026 (UTC)Reply
Wikipedia:Request_a_query#Vanished_Wikipedia_essay. I don't know SQL, so I can't do this myself. –LaundryPizza03 (dc̄) 04:41, 7 September 2026 (UTC)Reply
Replied there also ... but are you guys 100% sure you saw it on the English Wikipedia? Pages generally don't just vanish like that without a trace. I checked Gnomingstuff's central auth data, in case it was on Meta or something, but no dice. I've managed to restore weird old pages over the years; for instance, now I can't quite remember how I managed to find Wikipedia:Historical archive/List of administrators by edit count (deleted in 2007), but that was an interesting save. Graham87 (talk) 09:10, 7 September 2026 (UTC)Reply
I am fairly sure it was on enwiki. –LaundryPizza03 (dc̄) 19:10, 7 September 2026 (UTC)Reply
Is it possible that it was never deleted, just overwritten? Otherwise, if neither I nor Gnomingstuff contributed to it, which seems very likely, we have no idea where to look if it was in userspace. The title would have likely included "AI", "LLM", or "proofread". –LaundryPizza03 (dc̄) 19:27, 7 September 2026 (UTC)Reply
Or "copyedit". –LaundryPizza03 (dc̄) 19:58, 7 September 2026 (UTC)Reply
Speaking of vanished essays, I once saw a Wikipedia essay that quoted a user who threatened to place a "King's curse" on an editor they were in a dispute with, and I think the essay's point was that comments like that are silly to make. I don't remember the essay's title, so I don't know why it doesn't exist anymore. I asked about it at the Village pump and was told that a similar essay exists at Meta-wiki. – MrPersonHumanGuy (talk) 11:15, 12 October 2026 (UTC)Reply
How about MrPersonHumanGuy (talk · contribs)? They're the #1 editor to WP:AISIGNS according to page stats. –LaundryPizza03 (dc̄) 05:30, 8 September 2026 (UTC)Reply
There was also some pretty heavy refactoring of the AISIGNS page recently. ‑‑gurkubondinn 12:37, 8 September 2026 (UTC)Reply
Found it in this 17 July 2026 version of WP:USEDAI?, the last version that has the lumberjack table. Table was added by MrPersonHumanGuy, who is clearly skilled at hallucinating realistic LLM output, likely to be a critical skill in this emerging AI economy. –MadeOfAtoms (talk) 06:31, 8 September 2026 (UTC)Reply
Yay, nice one! Graham87 (talk) 07:06, 8 September 2026 (UTC)Reply
I've also mocked LLM output in this August 2025 comment at WT:AISIGNS and this October 2025 revision to Wikipedia talk:Sandbox. – MrPersonHumanGuy (talk) 10:38, 8 September 2026 (UTC)Reply
Maybe we should put this in its own essay, with actual LLM output? –LaundryPizza03 (dc̄) 12:14, 8 September 2026 (UTC)Reply
If imitations of LLM output are to be displayed alongside examples of actual LLM input, they should be incorporated into the prose (like how it's done at WP:YESNOLLM or WP:AIUR) so they don't get mistaken for actual LLM output. Fake examples should only be used to elaborate on how LLMs would write, not as examples of what they have written. – MrPersonHumanGuy (talk) 13:52, 6 October 2026 (UTC)Reply
I like the WP:GARAGE essay, which contains a fake mini-article (with footnotes) illustrating common elements in articles about non-notable bands. I thought a similar thing could be done for AI slop. ~Anachronist (who / me) (talk) 14:49, 6 October 2026 (UTC)Reply
There is Wikipedia:Canned user pages, which features a fake user page illustrating some common elements in user pages that follow a particular suspicious format. – MrPersonHumanGuy (talk) 15:42, 8 October 2026 (UTC)Reply
Coincidentally, I came here to ask whether this user page and their talk page are AI-generated. I am not sure what to do, other than adding their page to WP:Canned user pages. --not-cheesewhisk3rs ≽^•⩊•^≼ ∫ (pester) 21:31, 9 October 2026 (UTC)Reply
Ah, someone already added them there. LOL. --not-cheesewhisk3rs ≽^•⩊•^≼ ∫ (pester) 22:17, 9 October 2026 (UTC)Reply
Um. I'm a real user. IT person living in Lilongwe if that counts. Not sure if I've broken some sort of convention here..? XexFlare (talk) 05:45, 10 October 2026 (UTC)Reply
@XexFlare: I don't doubt that you are human. However, your user page looks AI-generated. I don't know if this is disallowed either, which is why I asked here (and accidentally pinged you, sorry about that). --not-cheesewhisk3rs ≽^•⩊•^≼ ∫ (pester) 07:16, 10 October 2026 (UTC)Reply
Yeah I barrelled into that one entirely by accident. Im very new to actually contributing and I just copied the format from other contributors in the Malawi projects. I've learnt today that I should avoid doing that going forward. I've since changed both pages. Can revise them down furthur if thats the convention for new contributors. Perhaps leave them as a stub for now? XexFlare (talk) 08:24, 10 October 2026 (UTC)Reply
It's basically not something someone is going to get sanctioned over, but it definitely means that there's an AI user that needs to be looked into if they've made other edits. Having AI-generated user pages doesn't disrupt the project, so they're usually left alone, but it's not a good sign. ‑‑gurkubondinn 08:32, 10 October 2026 (UTC)Reply
As far as I'm concerned, a user page that appears AI-generated (or resembles user pages that may be) isn't necessarily a bad thing, as long as you aren't pasting LLM-generated text into articles, drafts or comments and trying to pass it off as your own work. – MrPersonHumanGuy (talk) 11:59, 10 October 2026 (UTC)Reply
All good and apologies. I stepped on a rake with this one. I've since revised the user pages to make them sparser and removed the md entirely. I'm a new contributor assigned the Wiki task by a local group. I wasn't aware there was a convention I've broken. Now that I've addressed it can I get my entry removed from the canned user pages list? Or are there other steps I need to follow to achieve this? XexFlare (talk) 10:34, 11 October 2026 (UTC)Reply
I've removed it from the canned user pages list. --not-cheesewhisk3rs ≽^•⩊•^≼ ∫ (pester) 11:13, 11 October 2026 (UTC)Reply
@XexFlare: I think it would be useful to know whether you did use AI or not for your old user page. If you prefer not to say, that's fine since it's your userspace. But looking through WikiProject Malawi members, I only see one user page that looks slightly like your old one, which isn't much. --not-cheesewhisk3rs ≽^•⩊•^≼ ∫ (pester) 19:47, 11 October 2026 (UTC)Reply
Thanks for the question. Not directly no. But I think I was destined to crash into this problem at some point. In my field (IT/Code) use of headings, tables and especially md are bread and butter. We are actually pushed to look more 'AI' not less so it's a bad habit I've developed. Moving forward I will make a mental note to approach formatting differently on Wikipedia etc. And I welcome the rest of you keeping me honest in that regard. XexFlare (talk) 07:24, 12 October 2026 (UTC)Reply
It took me a moment to figure out that by md, you mean Markdown. – MrPersonHumanGuy (talk) 11:09, 12 October 2026 (UTC)Reply
If I were you, I would use {{no ping}} in my comments so I don't accidentally notify users that I've mentioned them whenever I link to their user pages. – MrPersonHumanGuy (talk) 20:00, 11 October 2026 (UTC)Reply
Yes, I noticed that this was what had caused XexFlare to be pinged (and accidentally pinged you, sorry about that). Thanks for the reminder - I'll be sure to use noping, and feel free to remind me again if I forget again. --not-cheesewhisk3rs ≽^•⩊•^≼ ∫ (pester) 20:19, 11 October 2026 (UTC)Reply
Your mention of the garage band essay has inspired me to create Wikipedia:Don't use AI to write about your garage band, which includes its own fictional example. – MrPersonHumanGuy (talk) 11:17, 12 October 2026 (UTC)Reply
Hey, that's good; the most egregiously extreme example of blatant LLM authorship. I like it! ~Anachronist (who / me) (talk) 14:19, 12 October 2026 (UTC)Reply
It's too good. My eyes reach the sample-article part and my hindbrain just screams "No! Do not read that!" M kuhner (talk) 15:19, 12 October 2026 (UTC)Reply

Backlog drive

[edit]

What do people think about the possibility of a backlog drive? I'd envision making some training materials and pointing people at tasks in some way. Looking at AINB now I see several classes of tasks from fairly easy to extremely difficult. I also see that we've got less than half the manpower needed to keep up with them: most cases are archived without resolution. M kuhner (talk) 17:34, 10 September 2026 (UTC)Reply

Hard to see what a drive would be defined by and people use enough GPTZero as is. I would be willing, however, to hold some sort of training camp if that is something people would be interested in. Fermiboson (talk) 18:32, 10 September 2026 (UTC)Reply
I think it would be a good idea given how undermanned WP:AINB is. Fermiboson has a point that it needs to be defined correctly since detecting AI-generated content can be tricky. One idea would be to focus on cleanup for articles where there is already consensus that they include AI-generated content. Phlsph7 (talk) 08:41, 12 September 2026 (UTC)Reply
For other examples of training programs, we have the New Pages Patrol School (WP:NPPS) and the Counter-Vandalism Unit Academy (WP:CVUA). The most obvious metric we could use for AI cleanup is the number of cases in Category:AI noticeboard open cleanup cases, which according to the category page is being tracked at https://dvrted.github.io/AINB-stats/ thanks to DVRTed maintaining the tracker. — Newslinger talk 12:42, 13 September 2026 (UTC)Reply
I floated this at at VPI but it didn't gain much traction. The issue with a backlog drive is I'm not sure we have the extra bandwidth necessary to tutor/onboard lots of people, though ofc if they're as competent as yourself and Purring then that wouldn't be a problem. I think below would help a lot, lots of experienced editors read the Signpost, and a call-to-action there would probably rope-in some people. If things go as planned and AINB becomes more manageable such that we can start chipping away at the backlog, I think a backlog drive focussed on tracking pages would then be worth pursuing, but imo we're just doing damage limitation atm Kowal2701 (talk, contribs) 14:54, 16 September 2026 (UTC)Reply

We are sort of cooked and there's nothing we can do about it

[edit]

Doomer title, I know, but refer to the attached log graph. That's the number of cases reported to AINB since its movement into project space in log plot. I don't remember the last time I saw a vaguely large case get significant cleanup attention before it getting archived. At this rate, AINB would reach 50 cases per day on February 19, 2027. We have, optimistically, 20 people around here who can do this every day and then not for more than a few hours every day. That's maybe enough to identify and tag all the cases, definitely not enough to do anything but mass reverting.

Yes, exponential growth can't continue indefinitely. But I encourage those of you registered as mentors to go check on your list of mentees. What's the percentage, out of the active ones, that are using AI? Every new editor we recruit, at this point, is 0.6 or 0.8 cases on AINB. And then we have the flood of sleeper NPPers and autopatrollers who somehow think it wise to trust their chatbot with their double digit year-old account. We will be overwhelmed in an exponential manner far before we reach a plateau. This is not even counting the amount of various bad actors that will become increasingly effective as LLMs improve.

Even if the entirety of the active editor population dedicated themselves entirely to AI cleanup, assuming the average AINB guest makes 1k edits (reasonable average between those caught early and the really old autopatrols), the rate of AI edits will exceed those of non-AI edits in articlespace - including semiautomated ones - on February 26, 2027. This is of course a massive overestimate of the time it will take as identifying slop is a lot slower than vomiting it, and fixing it is even slower.

I don't have some sort of magic solution to this. Ultimately it will probably involve some combination of automated filters and an OKA style external recruitment effort. Maybe watermarking will help. Maybe the AI crash will happen sooner than we think it will. Who knows. Fermiboson (talk) 19:31, 13 September 2026 (UTC)Reply

I'm pretty close to hacking together a plugin that will always display all wiki articles to me in the state they were in in 2021, with a button to show the diff and approve changes since. Essentially pending changes for the whole wiki and all users. That aside, I have a few words of encouragement:
  • AI text watermarking and improvement of AI detection tools may let us eventually add automated AI detection as an [Wikipedia:Edit check|edit check]] and automatically tag violations, allowing us to focus less on detection and more on cleanup.
  • Increasingly negative publicity about AI in the public discourse plus increasing visibility of Wikipedia's human-focused stance may reduce the number of incoming AI edits, at least from good-faith editors.
  • Other backlogs have grown exponentially for years and then reversed, for example I used to work on old POV disputes and over the course of several years I saw the backlog drop from nearly 10,000 articles to just over 2,000.
  • You say we have maybe 20 people working on AI cleanup every day, some for hours a day. What did that task force look like 2 years ago? I feels to me like the interest in helping with cleanup work has also grown exponentially, though it still lags behind the need.
Basically, soldier on comrade! There is no fate but what we make for ourselves.. -- LWG talk (VOPOV) 20:20, 13 September 2026 (UTC)Reply
I was just having a similar thought. Maybe there’s a checkbox on the edit screens. If you attest your content is organic it goes on the main branch. If the edit wasn’t certified organic it goes on a page level fork. Readers can choose if they want organic or sloppy. Dw31415 (talk) 20:28, 13 September 2026 (UTC)Reply
Related concept: Wikipedia talk:WikiProject Articles for creation#New field for AI use disclosure at AfC Submit Wizard? That change has consensus, code is ready, just waiting for an AfC script maintainer to review it. Dreamyshade (talk) 21:30, 13 September 2026 (UTC)Reply
But why would anyone tick the self-decline box? A vanishingly small minority of those caught pasting AI-output admit to it, most users just double down and swear up and down that there is no way that they used AI. The ones that admit to it are easy to work with, they usually just didn't know. ‑‑gurkubondinn 21:29, 14 September 2026 (UTC)Reply
AI text watermarking and improvement of AI detection tools may let us eventually add automated AI detection as an [Wikipedia:Edit check|edit check]] and automatically tag violations, allowing us to focus less on detection and more on cleanup.
Quick note to +1 the point I understand you to be making here, @LWG. Specifically, the idea of using tools like Edit Check to detect problematic edits and when warranted, deter/prevent people from saving them.
Assuming the above accurately reflects what you were getting at, two things:
1. By early October, you all can expect a new Edit Check, specifically designed for detecting when people are pasting text from LLMs, to be live on en.wiki.
2. Might there be other signals that you can see being helpful on the "detection" piece? Asked another way: if you could "magically" know certain things about an edit (or the person who made it) for the purpose of detecting AI-related infractions, what might they be? PPelberg (WMF) (talk) 22:48, 15 September 2026 (UTC)Reply
The biggest useful sign that can be quickly processed is AI edit summaries (see User:Fermiboson/AIlog). The tool is not accurate enough to make any sort of blocking edit filter over it but it is accurate enough that it is useful. I have tried to make a deterministic (regex/whatever) version of the tool and have not been able to achieve anywhere near the same performance. IIRC User:Dr vulpes has made something which can detect tone indicators though specific tone indicators generally become obsolete very quickly with time. LLM paste check is a big boon, but there are obviously LLM edits that somehow don't trigger pastecheck as is. Otherwise, there are a lot of LLM translations or copyedits that are only detectable through meticulous source-text verification, machine assisted or otherwise. Maybe some combination of the more deterministic factors like WP:AIREDCAT could be added to the EF. The difficulty is that whatever detection method we use, the symptom itself is allowed and sometimes done by humans, so we cannot use a disallow edit filter. We would still have to have humans look over filter logs or paste check tags or whichever detection tool we end up using. Also, the rate of incoming LLM edits is likely to increase in a manner such that even if we had a detection tool which could keep up with all live edits and had a 0% FP, we would still lack the volunteers required to actually do the cleanup unless we're just blanket reverting everything. Fermiboson (talk) 23:04, 15 September 2026 (UTC)Reply
Seconding that if we can't block LLM edits from being published, we would still lack the volunteers required to actually do the cleanup. Kowal2701 (talk, contribs) 12:42, 16 September 2026 (UTC)Reply
There also needs to be some indication that a user's edits have been blocked, otherwise it's just delaying detection and helping users hide it better. ‑‑gurkubondinn 12:44, 16 September 2026 (UTC)Reply
@PPelberg (WMF), there's some trickery that @Asilvering does as a CU that can identify agents, idk if that's automatable. I'm not aware of any hidden info which would be useful to know wrt human editors.
We already have Special:AbuseFilter/1325 which is based on strong AI signs wrt prose, but it gets too many false positives to be worth patrolling. That's based on only one sign being present though, when what makes AI-generated content detectable by humans is the presence of multiple strong AI signs (i.e. artefacts which rarely appear in human writing, multiplying the already-low probabilities rapidly increases certainty). WP:AISIGNS is pretty rudimentary and outdated atm. It may be possible to automate this, but systematically determining AI signs for different time periods requires processing a hell of a lot of data and resources (though once done I assume the results could be integrated into Edit Check?). Gnoming already has databases which should prove valuable if this a route we want to go down. The cost-benefit ratio sort of depends on how (in)effective (or even feasible) automated watermark checking will be.
Is there a central log for Edit Check? That would be hugely useful. I can see there's Special:EditChecks for configurability. There's Special:Contributions where you can search for the tag, but you have to input a username Kowal2701 (talk, contribs) 12:40, 16 September 2026 (UTC)Reply
(pinging @M kuhner here, mostly for her peace-of-mind lol) Kowal2701 (talk, contribs) 14:49, 16 September 2026 (UTC)Reply
Thanks!
Probably off topic here, but the exception from NOLLM for translation is a very large issue. I would strongly support trying to get rid of it. LLM translations introduce hallucinations and bias just like LLM generations, plus we are seeing a flood of "translations" which take maybe a para from an other-wiki version, fill the rest with slop, and claim the exemption. We are increasingly taking a hard line on LLM translations but since they are formally allowed, have to spend extra time. I've seen at least two cases disregarded because "they're translations" and no one has time for that. M kuhner (talk) 15:09, 16 September 2026 (UTC)Reply
Yeah, I agree. Right now I'm sort of hoping that that'd be a happy accident of , and if desired we can create a user right for LLM translations where the holders' edits are whitelisted from the 'firewall' Kowal2701 (talk, contribs) 15:31, 16 September 2026 (UTC)Reply
@Kowal2701, that is in fact already automated - the flagging, anyway. All the checks and blocks are humans, of course. @PPelberg (WMF), if I recall correctly it was @MSzwarc-WMF who developed this one, starting from a request and some samples I gave to PSI. I think that's about as much as I can say about it publicly, but you know how to talk to each other. :) In solidarity, asilvering (talk) 15:45, 16 September 2026 (UTC)Reply
Thanks @PPelberg (WMF) for reaching out here. As I see it there's two ways automated detection tools can help us:
  1. targeting user-education by popping up a message informing them of WP:NOLLM when the system detects insertion of LLM text.
  2. targeting cleanup by automatically tagging diffs that the system assesses as containing LLM text.
the first use is what an edit check can do for us as I understand it. It sounds like you are saying there is some way to log edits where the user received an edit check warning and chose to proceed anyway? That would also help with the second use. I agree with others that an edit check that disallowed saving entirely when LLM text was detected would be counterproductive since it would just coach people through obfuscating their text before saving. As automated AI detection becomes more reliable I'm hoping to eventually get an interface like the Who Wrote That plugin that will highlight suspected AI text in-article and show the responsible editor on hover. A tool like that would really speed review. -- LWG talk (VOPOV) 13:06, 16 September 2026 (UTC)Reply
I don't know the technical details behind LLM paste check, but if triggering it indicates 100% certainty it was pasted from an LLM session, we might want to consider setting it downright to disallow. Fermiboson (talk) 14:20, 16 September 2026 (UTC)Reply
+1, AFAIK it goes off of metadata and has no false positives (?) Kowal2701 (talk, contribs) 14:21, 16 September 2026 (UTC)Reply
It's a little fuzzy. We're currently looking for signs in the pasted content that seem to be fairly distinctive. Some of them are more certain than others, but none of them are 100% that it came from a given LLM.
The challenge is that all we have to go on is a generally-short snippet of whatever came off the clipboard -- we explicitly don't know e.g. what URL the copy came from. Sometimes if you've copied little enough, or the user chose to paste without formatting via the OS shortcuts, it'll just be plain-text and we can't tell anything.
What we're not doing is any sort of SynthID-esque analysis of the contents for a watermark that's contained within the text itself. There's two problems with doing that: first it generally requires a larger amount of text, and second all the current systems rely on a secret that the content-generator knows and so needs an API call to check (which isn't actually available yet for Google if you're not an enterprise partner).
Overall: if it triggered I'd think it's quite likely we're right, but not certain. And we're definitely going to miss some. DLynch (WMF) (talk) 17:11, 16 September 2026 (UTC)Reply
I thought it was just a paste- check at first with hope that future watermarking might be able to add actual detection.https://www.mediawiki.org/wiki/Edit_check/Paste_Check Dw31415 (talk) 16:50, 16 September 2026 (UTC)Reply
Thank you for making this. Has anyone made a chart of the number of articles in Category:Articles containing suspected AI-generated texts, and user pages in Category:User talk pages with large language model notices, over time? I would love to see all three charts in a Signpost essay, along with the New Pages Patrol backlog chart (I suspect that NPP also being overwhelmed means that some of the slop that could have been caught immediately wasn't). I just want more people to see the magnitude of what's happening.
Tiny related idea: could it be appropriate or helpful to put a call-to-action near the top of Wikipedia:Signs of AI writing, asking editors to help with AINB? It's a pretty visible page. Seems like it can't hurt. Dreamyshade (talk) 21:52, 13 September 2026 (UTC)Reply
Have a nicer version (vibecoded matplotlib, ironically) of the above graph - linear and log plots. On closer inspection I would actually argue the data shows superexponential growth.
The others are harder to do with a simple quarry query. LLM tagging is apparently mostly not done with twinkle or pagetriage and Gnomingstuff uses a lot of varying edit summaries that I can't easily track. I am also not convinced that it reflects anything due to varying LLMPROD deletions/text removals/completed cleanups/etc. For the user talk pages, apparently a considerable portion also don't use Twinkle, but enough of them do that I'm comfortable publishing this - it counts every time someone posts a notice, including on pages already with notices, because each posting is a workload. Caveat is that this growth appears to be subexponential.
Fermiboson (talk) 12:55, 14 September 2026 (UTC)Reply
May I ask, I would be interested in making that 21 people who can help out. I don't have mentees and I don't plan to. But can I attack the backlog directly, and how? I've only just now filed my first report at WP:AIN, so if you have a help guide explaining the process, I'd be happy to take a look. I also worry that this will burn me out much quicker than even WP:AfC, but I hope I know how to step back if I need to... guninvalid (talk) 07:42, 14 September 2026 (UTC)Reply
@Guninvalid I have been picking a page on Category:AI noticeboard open cleanup cases and working my way through the articles flagged for cleanup. I’m not fast at it yet. Sometimes it’s easy, sometimes it’s just tiring and upsetting, but it’s something I can plug away at in my spare time and I hope it’s helpful. Suðurhafsljósæta (talk) 08:58, 14 September 2026 (UTC)Reply
To shamelessly self-promote, I'm running the edit summary detection bot in the AIC discord server now, and it would be helpful as ever to have more pairs of eyes checking the edits flagged there. In my experience 40-60% of the flagged edits have something to do with LLMs and new cases are very useful for honing your detection skills, both because they keep you up to date in a rapidly evolving field and because you don't need to care about cleanup - just click the revert button. Fermiboson (talk) 09:40, 14 September 2026 (UTC)Reply
If there’s a wiki project you like, I can add a report of that projects pages with AI. Dw31415 (talk) 11:34, 14 September 2026 (UTC)Reply
My version User:Dw31415/AI Cleanup Dw31415 (talk) 11:50, 14 September 2026 (UTC)Reply
@Guninvalid, there's WP:AICGUIDE, lmk if you have any questions Kowal2701 (talk, contribs) 13:28, 14 September 2026 (UTC)Reply
Thanks for the stats. Unfortunately the first thing I thought was that I don't report all the cases I see. Still, this is clarifying perspective. CMD (talk) 13:39, 14 September 2026 (UTC)Reply
yeah, same, I try to handle as much off-noticeboard as possible Gnomingstuff (talk) 04:50, 16 September 2026 (UTC)Reply

Invitation to write an essay for The Signpost

[edit]

Thanks @Dreamyshade: for suggesting a story about this for The Signpost. I am an editor there, and I would support a submission. I confirm that readers have been reactive to stories on the effects of AI, and also, if anyone wanted to make a push for more people to participate in this project, or to ask for other resources from the broader Wikimedia Movement, then anyone could use the essay an opportunity to recruit volunteers or describe a problem in need of more and other support. The submission could take any format, and for a story like this, there could be one or multiple submissions. In any case, start by posting a 1-2 sentence idea at Wikipedia:Wikipedia_Signpost/Newsroom/Submissions, so that both Signpost editors and anyone else from here who might want to contribute can be aware of efforts in development. @Fermiboson: Thanks for making the visualizations and starting this framing of the discussion. Those would make a great addition to an essay. You already have you contributed the graphs, ideas, and organization, and I would not pressure you further to also write an article, but you certainly are invited to do so. For everyone else here, no one can tell this story alone, and stories like this are more powerful when small groups of people can reach consensus on some key points and recommendations after collectively interpreting the available information. Develop the story however you like, but if it were possible for a group of people to speak as "WikiProject AI Cleanup" or a public member taskforce of it, then that could give me message more weight. Signpost just published today and issues come out every 2-3 weeks. There is no deadline. However, if and when anyone does make a submission, then I can advance that to the other editors there for review, and seeing the draft in place sooner might make it easier for anyone here who wants to contribute to do so. Thanks. Bluerasberry (talk) 14:35, 16 September 2026 (UTC)Reply

Thanks - I will see what I can get up to. Fermiboson (talk) 16:01, 16 September 2026 (UTC)Reply

A user essay I just cooked up

[edit]

As a subtweet of the above discussion, and in response to the thread WP:AIN#User:1keyhole, I cooked up a user essay at User:guninvalid/It doesn't have to be AI to be slop. I'm curious to see WP:AIC's viewpoint on this advice. But the abstract is that proving AI specifically is often an uphill battle or outright impossible. But even if it isn't AI, it's still wrong and bad and in need of cleanup. guninvalid (talk) 08:59, 14 September 2026 (UTC)Reply

I don't think that's helpful, we have enough cases where people deflect or are evasive, and at the 1keyhole case your judgement was wrong wrt AI use (you also lectured Gnoming, who is like our #1 expert at detecting AI-generated content). The most productive thing for people at AINB to do is to determine whether there was AI use and how pervasive it was, people should report to there because we have the expertise and the measures to efficiently clean it up thanks to WP:LLMPRV (other than that, WP:PDEL and WP:BANREVERT, presumptive removal is practically never invoked at enwiki). Kowal2701 (talk, contribs) 16:29, 14 September 2026 (UTC)Reply
I think it was just a misunderstanding not a lecture and I was overly rude about it, apologies again Gnomingstuff (talk) 17:03, 16 September 2026 (UTC)Reply
The section section is a helpful reminder for users with very few edits or who are inactive - if their content is obviously low quality you don't have to demonstrate AI use before removing and sometimes it's better to just choose the most efficient way to get the content out of mainspace. For users who are continuing to add more AI content its important that they understand that the AI use itself is a problem. If you focus on other issues with the content you set them up to just tweak their prompts and continue until they get blocked. -- LWG talk (VOPOV) 17:24, 14 September 2026 (UTC)Reply
I agree that making arguments based on the quality of the content is important, and I often do it, but there's one thing that's troublesome - there's no such thing as "presumptive WP:V failure" or "presumptive WP:SYNTH", so once you give up on treating the AI signs as AI signs and instead devote yourself totally to arguing content in a particular case, you lose the ability to do substantial cleanup and instead have to pick your battles based on which of the editor's articles you want to spend hours analyzing and either rewriting it yourself or arguing every individual point with the editor. When you have a case with hundreds of affected articles, especially articles that already existed and have notable subjects so can't be deleted, that makes the scope of the work even more insurmountable than it already is. purringmaggot (talk) 18:40, 14 September 2026 (UTC)Reply
Sorry, but I gotta say that I'm with Kowal here. ‑‑gurkubondinn 21:20, 14 September 2026 (UTC)Reply
This essay shows an interesting shift. The reason the AI restrictions came in was because AI use was the Al Capone tool that allowed for the saving of time vs investigating each AI article for its hallucinations etc. Oddly, perhaps this is a sign that our processes relating to AI have improved, and we forget how frustrating the before times were. CMD (talk) 23:49, 14 September 2026 (UTC)Reply
This seems to have been met with a mixed reception. I see some consensus that this advice is worth taking into account at least sometimes, but I acknowledge that I only know what I know. So I'll keep it as a user essay. Given this reception, I no longer wish to try to promote to WP space. If you find this advice useful, great! If not, also great. guninvalid (talk) 08:26, 16 September 2026 (UTC)Reply

AI cleaner

[edit]

In case any users of User:Sophisticatedevening/AI Cleaner.js start wondering where it went, I've disabled the script following a request on my talk page because it is being misused to mask AI text. A very unfortunate side effect but evidently necessary (although the problems it was being used to address aren't as common as it was when I made the script anymore). Sophisticatedevening(talk) 20:38, 14 September 2026 (UTC)Reply

Aura (company) AI content

[edit]

Hi editors, I'm Jory with Aura (identity management company). An article was created about the company that seems largely AI generated and one section in particular has been flagged by another editor as likely written by AI. I have posted a COI edit request with alternative wording for the section that is 100% human written. It may be of interest to editors here. Thanks! AuraJory (talk) 18:31, 15 September 2026 (UTC)Reply

 Done purringmaggot (talk) 01:01, 16 September 2026 (UTC)Reply

Using Pangram

[edit]

When pasting article text into Pangram, is it better to use the text as displayed to the reader or the wikitext that people using the source editor would see? Also, should you keep the referencing syntax (which in the former scenario would just be the reference number in brackets, and in the latter would be the actual wikitext that inserts a reference) in or leave it out? SwineHerd (talk/contribs) 22:45, 15 September 2026 (UTC)Reply

neither imo, paste the text as displayed to the reader, then remove all the refs and headings so it's just prose Kowal2701 (talk, contribs) 14:59, 16 September 2026 (UTC)Reply
I use User:Anne drew/CopyPlainText for this purpose. I don't know for sure that it's optimal, but I'm guessing that the engine was trained on text rather than markup, so it seems reasonable to use the plain text to check. Dreamyshade (talk) 21:25, 16 September 2026 (UTC)Reply

Help improving MoveToDraft messages

[edit]

As New Page Patrollers attempt to chew through the unreviewed backlog, we're draftifying a lot of articles, and I'm not sure that the default user talk messages provided by the commonly-used MoveToDraft userscript are helping editors learn about what they did to cause the draftification, especially regarding WP:NOLLM, or what they'd need to do to revise the article before moving it back to mainspace. For example, this is a draftification message that I generated via the script:

"Thanks for your contributions to Smart Arena Events Platform. Unfortunately, I do not think it is ready for publishing as a live article at this time because it needs more sources to establish notability, you may have a possible Conflict of Interest and it consists of machine-generated text. I have converted it to a draft which you can improve, undisturbed for a while. Please see more information at Help:Unreviewed new page. When the article is ready for publication, please click on the "Submit the draft for review!" button at the top of the page OR move the page back."

I wrote a small suggestion to the userscript maintainer to add policy links, but thinking about it, I suspect these templated message might really benefit from deeper rewrites (while attempting to stay brief) to ensure people are properly informed of WP:NOLLM. So, sharing here in case people have other ideas. [Courtesy ping to User:MPGuy2824 because I'm talking about your work.] Dreamyshade (talk) 21:41, 16 September 2026 (UTC)Reply

Thank you User:MPGuy2824 for updating the template messages! I'm hoping that the message of "it consists of AI-generated text" is at least slightly more helpful for editors.
Related: I added a new segment to Help:Unreviewed new page about "AI-generated content", because it didn't have one yet. Please feel free to review and refine. Dreamyshade (talk) 03:13, 19 September 2026 (UTC)Reply
Somewhat relatedly, I think the uw-ai series could use some work, especially the later ones in the series ("your edits have been reverted"). A lot of the time the boilerplate isn't and can't be accurate -- for instance, if someone mostly creates new articles. Gnomingstuff (talk) 16:56, 19 September 2026 (UTC)Reply

Discussion at WP:VPT § LLM Paste Check - live later this week

[edit]

 You are invited to join the discussion at WP:VPT § LLM Paste Check - live later this week. Quiddity (WMF) (talk) 00:32, 23 September 2026 (UTC)Reply

For convenience, here’s a link to recent changes with the tags https://en.wikipedia.org/wiki/Special:RecentChanges?days=30&hideWikibase=1&hidebots=1&hidecategorization=1&limit=1000&tagfilter=editcheck-llm-paste-shown%7Ceditcheck-llm-paste&urlversion=2 Dw31415 (talk) 04:12, 27 September 2026 (UTC)Reply

Another AI agent

[edit]

At Special:Contributions/Owlmy blocked after a hilarious bot loop, for your perusal and entertainment. Fermiboson (talk) 07:31, 29 September 2026 (UTC)Reply

That account probably needs a global lock or something, it has been going haywire on Wikidata as well. ‑‑gurkubondinn 07:48, 29 September 2026 (UTC)Reply

Userscript checking for hallucinated sources

[edit]

I've made (vibecoded) a userscript which checks for hallucinated and nonexistent sources at User:Fermiboson/sourcecheck. Questions, suggestions and advice are welcome. Fermiboson (talk) 04:39, 30 September 2026 (UTC)Reply

Message to user after Paste Check

[edit]

I reached out to this editor after a recent edit that triggered paste-check. Did I to that right? See User talk:Steelcity18#LLM use on Coastal Carolina–Liberty football rivalry Dw31415 (talk) 18:53, 30 September 2026 (UTC)Reply

AI use in hospital and medical settings

[edit]
 – Moved discussion to appropriate page Dw31415 (talk)

In the magazine for medical transcriptionists (Association for Healthcare Documentation Integrity) there was an articicle regarding AI making its own decisions regarding what medications to give and what the plan should be for the plan. This caused a lot of harm to the patient. Do you have any further articles regarding AI use in hospital or medical settings? Klewis1119abcd (talk) 16:28, 3 October 2026 (UTC)Reply

Welcome to Wikipedia! Thanks for registering and making your first edit in this account. This talk page is where we try to address editors adding LLM generated content to articles, which is not allowed (see Wikipedia:NOLLM if your interested). I looked for existing content on the topic. I'd expect it at SaMD, but didn't see anything. Let me know if you find anything. You could try expanding the last paragraph in that section with the article you found (either directly edit the article of raise a question the talk page Talk:Medical software. I'll ping you there now so we can move the discussion from here. Dw31415 (talk) 16:58, 3 October 2026 (UTC)Reply

Edits with a page move mixed in

[edit]

I am currently working on cleanup for WP:AINB#Pristino. In a handful of cases the edit history has a bunch of Pristino edits, a Pristino page move, and a bunch more Pristino edits. I am reluctant to rename a page as part of cleanup. What's the best strategy for these? Currently I just swear and tag them. M kuhner (talk) 15:30, 14 October 2026 (UTC)Reply

For any pages where you're not sure if the move-back is appropriate, I would nominate them for WP:RM back to their original title with a short explanation like This page was moved to X by User:Pristino; Pristino's edits to this page have been reverted via presumptive removal as their edits were made using LLMs in contravention of WP:LLM (Wikipedia:AI noticeboard/Pristino) Wracking talk! 03:24, 15 October 2026 (UTC)Reply
I don't think I understand this answer, sorry. Can you lay out a step by step of what you mean?
M kuhner (talk) 04:07, 15 October 2026 (UTC)Reply
I'm sorry my first response was confusing. If you're not willing to rename pages as part of the cleanup, or not sure if they should be renamed, I would recommend sending them to Wikipedia:Requested moves so other editors can weigh in.
To nominate a page to be moved, add this to the bottom of the talk page of those articles (replacing the page names as needed):
{{subst:Requested move | ''pre-Pristino page name'' |reason= This page was moved to ''post-Pristino page name'' by [[User:Pristino]]; Pristino's edits to this page have been reverted via [[WP:LLMPROD|presumptive removal]] as their edits were made using LLMs in contravention of [[WP:LLM]] ([[Wikipedia:AI noticeboard/Pristino]])}}
(You can also use WP:TWINKLE for RMs; it's under the XFD sub-menu.) Wracking talk! 06:28, 15 October 2026 (UTC)Reply
This is the opposite of the problem I have, though. If I clean up Pristino's edits in the normal way I will revert to pre-Pristino page name as a side effect, by undoing their move. Is it okay to do that without investigating the reason for the rename? Should I worry that the other name may be taken now? (I.e. does a disaster happen if I find article Y, revert it to X, but in the meantime X has been created?) Will I break the Talk page? I have the heebie-jeebies here.
If I leave the move standing it becomes very hard to get rid of the pre-move Pristino edits: I'd have to hand-edit them out of the text one by one, because restore version and rollback can't be used. M kuhner (talk) 06:34, 15 October 2026 (UTC)Reply
Could you provide an example of an article with this issue? Reverting an article to its state prior to a page move does not move the page. (Just to check, you're talking about page moves, right? Not splitting or merging?) Wracking talk! 06:43, 15 October 2026 (UTC)Reply
Okay, that was what I didn't know! Quite a different situation then. Thanks, that solves my problem. I had assumed that reverting a move unmoved it. M kuhner (talk) 06:48, 15 October 2026 (UTC)Reply

Do we have an edit behavior anomaly detection model?

[edit]

A discussion that I was involved with was tainted with large-scale apparently agentic votes (see here). Thanks to the effort of the community, it was eventually caught. I'm wondering if we already have some automated anomaly detection tools that may make lives easier for us. If not, should we create one? Redwolf Milkshake Fox 🦊 02:57, 15 October 2026 (UTC)Reply

I’d like to hear others’ thoughts on this comment I made at the AI Noticeboard discussion on this. I’ve encountered other presumably agentic editors in the past that could have reasonably been earlier iterations / other attempts at that exact sort of thing.
Best,
CSGinger14 (talk) 03:07, 15 October 2026 (UTC)Reply
I think more automated LLM-like-behavior detection is a good start toward countering this kind of thing. We wouldn't need to use ML; a deterministic model would probably work well enough. For example, start by looking for users who add more than a certain number of bytes per minute. Maybe lower the threshold if a lot of them have editcheck tags. If we're concerned about false positives on humans pasting the same text into multiple articles, we could even set it up to check for and ignore identical text on different articles. Justin Kunimune (talk) 13:20, 15 October 2026 (UTC)Reply
I’ve been thinking we need to rate-limit new users or something like that. (Only ~25 edits per day to the article space, etc). Has this been discussed yet? Dw31415 (talk) 15:57, 15 October 2026 (UTC)Reply
I'm more worried about bad actors creating clandestine agent fleets under different user accounts to slowly tweak articles, flying under the radar. I'm thinking of risk models similar to those fighting financial crimes. Redwolf Milkshake Fox 🦊 16:03, 15 October 2026 (UTC)Reply
If someone or some party goes to that level of magnitude to gradually increment changes with automation in a manner borderline undectable... you're talking about user accounts that would have to be created under the radar and at different dates/times. They'd have to do wholly non-AI detectable modest little housekeeping stuff. A month, three months, six months later they're sitting on genuinely as presented helpful edits, each north of 200, 300, 500 contributions. Just boring harmless isolated gnomes "as seen". The accounts we shouldn't look twice at.
The proposition is then that they iterate, as more like a slow moving front (an edit a week here, two there, from wholly unrelated on the surface users) to... push POVs? COIs? Violate WP:PAGs (ignoring the AI/LLM related ones a moment). Edits that... again, "as seen" would never and should never trip wire a non-paranoid individual, if done "right".
How would you even detect that, if someone went to that level of command and control design with that level of sophistication? Bluntly, you'd need either a data analytics/science wizard who can suss out some manner of correlation (all accounts launched within some N number window of months) or some other obvious gotcha tie-in. But then if they're designed (easy enough with LLM again) to NOT cross wires and even have just different enough personalities and schedules... again, you'd only catch it with a big brain group of people, or sufficiently designed and powered automated (LLM) analytics.
And given the disdain to apparently even use/acknowledge LLMs from the folks here most likely to be on the hunt for such an actor, good luck, I suppose. If it's state level, have fun with that.
I'm just getting into the weeds here to illustrate why I don't think dark horse type scenarios like you describe are worth considering much in policy or in decisions like this. You can plan for a planetary defense against meteors and such if you plan and spend for it. We could watch the entire sky if we wanted. But if some jackass somehow deploys 1000 autonomous thrusters to an asteroid somewhere and figures out a blind spot approach to drop a rock on Madrid by striking during a rare eclipse or something... there's only so many possible scenarios you can guard against. — VPP (t/c) 16:13, 15 October 2026 (UTC)Reply
To be honest, I don't think that solves a problem so much as it shifts it elsewhere. For example, we may see new editors suddenly turning to other namespaces - and if they were disrupting in mainspace where it would be easily caught, this would just shift the target of the disruption. – Epicgenius (talk) 17:37, 15 October 2026 (UTC)Reply
The one thing that's good about AfD is that enough eyes are on it every day that eventually something will get picked up. I think our biggest concern is the mainspace because of how expansive it is. If nothing else, I think we need to make this a sitewide discussion, because it's increasingly evident that this could have catastrophic consequences if the models get only a little bit better, and we come up with no means of defense, whether it be fighting fire with fire or not.
Additionally, I think that it might be good to look at the strategies used by sockpuppets as a general indication of effective strategies. We'd need to be extremely careful about falling into survivorship bias, but it's a good place to get a general idea of where their creators might be getting their inspiration from. See the pattern of edits by User:Jørgen Rü as an example, who I recently uncovered as a sock (unintentionally, funny enough) a few months back, and who had seemed to be covering their tracks quite well up to that point.
CSGinger14 (talk) 20:19, 15 October 2026 (UTC)Reply

AI noticeboard overhaul

[edit]

 The AI noticeboard has been overhauled and streamlined. A summary of the changes is available on the noticeboard's talk page, and you are invited to ask any questions there. Chaotic Enby (in solidarity · talk · contribs) 15:39, 15 October 2026 (UTC)Reply