Wikipedia talk:Large language model policy

RFC

The following discussion is an archived record of a request for comment. Please do not modify it. No further edits should be made to this discussion. A summary of the conclusions reached follows.

This RfC seeks to establish a policy or guideline on how Large Language Models (LLM) can be used on Wikipedia. There is no consensus to adopt any wording as either a policy or guideline at this time with most editors participating falling into one of three groups.

The largest group of editors was in favor of adopting the proposed wording as either a policy or guideline. The smallest group of editors proposed a complete ban of LLM use in mainspace, but with more flexibility for use in other spaces (what came to be called option 1b). The remaining editors opposed the wording proposed in the RfC. The primary objections from those opposing included concerns about mandatory disclosure and about the proposed ability to summarily remove suspect LLM content, including those who feel there is no accurate way to identify LLM generated content. Some of those in favor of the proposal did note that we have other policies and guidelines where detection of violations can be difficult, though some of those in favor of the wording overall also opposed summary removal.. There does seem to be an implied consensus for Large language model output, if used on Wikipedia, must be manually checked for accuracy (including references it generates) among those both favoring and opposing this wording but this was not stated explicitly enough by enough editors for me to formally find a consensus for it. Nothing in this close should be construed to suggest that current policies and guidelines do not apply to Large Language models, with a number of editors explicitly noting (especially among those opposing) that current policies and guidelines do apply.

Housekeeping: As an RfC to establish policy, I used no weighting of any arguments when determining this consensus nor did I use any AI other than what is built into GoogleDocs to help with spelling and basic proofreading. Barkeep49 (talk) 20:47, 19 January 2024 (UTC)

---

Per a prior RfC at WP:LLM, should the following sentence be adopted as a policy/guideline (whether or not it is expanded later, or whether or not there are subsequent decisions with respect to the use of LLMs)?

Large language model output, if used on Wikipedia, must be manually checked for accuracy (including references it generates), and its use (including which model) must be disclosed by the editor; text added in violation of this policy may be summarily removed.

• Option 1: Yes (specify "policy" or "guideline").
• Option 2: No.

jp×g🗯️ 22:22, 13 December 2023 (UTC)

Survey

• Option 1, as a policy. I wrote the first version of WP:LLM on December 11 last year, as a minimalist summary of existing guidelines and how they applied to LLMs; over the intervening months it was expanded greatly, adding many complex guidelines, prohibitions, recommendations, et cetera. The resultant doorstopper was then proposed for adoption, in an RfC that many of the page's own contributors (including me) opposed. This did not need to happen: we could have just written something concise and minimal, that everyone was able to agree on, and then workshopped modifications and additions. But as a consequence of the way this RfC happened, we do not really have any policy that governs the use of these models, aside from the obvious implications of existing policies (i.e. WP:NOT, WP:V, WP:RS, WP:C, WP:NPOV). Some people have attempted to expand other policies to cover LLMs, like WP:CSD (where a new criterion for unverified LLM slop has been rejected) or WP:BOTPOL (where it doesn't really apply, since LLMs are not being used to make unsupervised edits). I don't think it makes sense to try to write LLM policy by scrawling a couple sentences in the margins of a different policy -- the same way it doesn't make sense to regulate cars as a special unusual breed of horses. They're a pretty significant technology that will be increasingly ubiquitous as the future occurs, so we should make a policy for them, and we should start with something simple and straightforward. jp×g🗯️ 22:39, 13 December 2023 (UTC)

  This edit of yours has a 37% chance of being undeclared LLM. Do you think it should be considered "text added in violation of this policy" and therefore subject to summary removal (i.e., removal without discussion or an opportunity for you to claim that you weren't using LLM after all)? WhatamIdoing (talk) 01:26, 14 December 2023 (UTC)

  I feel like, even under the policy of the text as stated, there wouldn't be a reason to remove it. After all, the detector implies that the content is significantly more likely human than not. — Red-tailed hawk _(nest) 03:21, 14 December 2023 (UTC)

  Under the proposed policy, my stated belief (genuine or otherwise) would apparently be sufficient for immediate, un-discussed removal. WhatamIdoing (talk) 03:21, 17 December 2023 (UTC)

  I share WhatamIdoing's concerns about the difficulties of identifying LLM output. This was already discussed several times at WT:LLM without encountering a satisfying solution. Some editors in the discussion here have pointed out that the policy is not intended to start a witch hunt of edits that score high on AI detectors and have expressed their faith in the common sense of their fellow editors. However, given that editors do not have much experience with this new and rapidly evolving technology and that opinions on it are quite divided, I'm not sure that this faith is well-placed. This does not automatically mean that we should not have a policy/guideline but if we do, it should explicitly mention the difficulties associated with identifying LLM output since many editors may not be aware of them. Phlsph7 (talk) 09:55, 17 December 2023 (UTC)

  If this passes, I plan to immediately open an RfC on the specific methods of determining LLM authorship. Most "detectors" available online (specifically "GPTZero") are basically woo; no source code, no model weights, nothing besides some marketing copy on their website. The "technology" page on their site has a nice graphic with some floating cubes, and says that it has "99%" performance. Performance on what? A dataset that they... made themselves. lmao. I don't think it is appropriate for us to be relying on stuff like this to make anything resembling official determinations of guilt. jp×g🗯️ 00:19, 15 December 2023 (UTC)

  Maybe this RFC is putting the cart before the horse, then? You're trying to get a rule set that says "text added in violation of this policy may be summarily removed", and you provide neither any way for editors to determine whether that's happened, nor any recourse for innocent victims of bad judgment (or POV pushing by someone willing to lie about it seeming like LLM-generated text in their personal opinion). WhatamIdoing (talk) 03:19, 17 December 2023 (UTC)

  I am not sure identification is its own problem: if content removed as LLM turns out not to be, it can be discussed and readded like every other minor content dispute. If someone is lying or acting in bad faith about their LLM contributions, they can be handled via existing means. Remsense留 17:53, 23 December 2023 (UTC)

  We only have your word against mine that something is LLM generated (or that something isn't LLM generated). Sohom (talk) 18:42, 23 December 2023 (UTC)

  That's the case with plenty of content disputes. (And in all cases, there's also the text itself.) Remsense留 18:44, 23 December 2023 (UTC)

• Yes as a policy. Fermiboson (talk) 22:29, 13 December 2023 (UTC)

  @Fermiboson, do you wish to support the proposal as a Policy or as a Guideline? You may have missed that there are two suboptions in Option 1. :) TROPtastic (talk) 01:00, 14 December 2023 (UTC)

  Thanks, I have clarified it. Fermiboson (talk) 09:09, 14 December 2023 (UTC)

Notified WikiProject AI Cleanup. Queen of Hearts ❤️ (no relation) 22:30, 13 December 2023 (UTC)

• I don't feel the proposed guidance should be given the status of a guideline or policy. I think its focus on a single technology makes it overly specific. I feel that community consensus will be more easily discernable by categorizing software programs/features into types based on use, and gauging community opinion on these types. isaacl (talk) 22:33, 13 December 2023 (UTC)

  @Isaacl: I think that technologies substantially different from others should be governed by separate policies. For example, templates and articles are both text files written with MediaWiki markup using the same program, but we use separate policies to determine what's appropriate in each, because there are different considerations (for templates it's the execution of code and the graphical display of information, for articles it's organization and content). jp×g🗯️ 22:47, 13 December 2023 (UTC)

  I think there is more community consensus to be found regarding types of uses of technology. The same technology often underlies a plethora of uses. No one's concerned about the technology used by grammar checkers. I think the community is concerned about programs generating article text, regardless of the technology used by the programs. isaacl (talk) 23:08, 13 December 2023 (UTC)

• Yes, as a guideline. Cremastra (talk) 22:56, 13 December 2023 (UTC)
• Option 1 policy, something is needed to prevent substitution of human accuracy and verification, especially non-English speakers who use LLMs to communicate and write on the English Wikipedia without knowing what they are trying to say, and guidelines are insufficient for this. — Karnataka ^talk 23:29, 13 December 2023 (UTC)
• The proposed language sounds reasonable to me, although I'd like to see some more discussion/what those more skeptical might raise as counterarguments before making a bolded !vote. I do want to note now, though, that if I support, it will be as a guideline, not a policy. Quoting from Help:PG, Policies express the fundamental principles of Wikipedia in more detail, and guidelines advise how to apply policies and how to provide general consistency across articles. Our caution around LLMs is something that we're adopting more because it makes sense given our goals, rather than because it's a fundamental principle, so a guideline would be a more appropriate formulation. To put it another way, if a non-Wikipedian were to ask me, "What are the fundamental beliefs that Wikipedians hold?" I might talk about verifiability or neutrality or any other number of policies. I would not ever think to say, "Caution around LLMs," so it would not make sense to formulate this as a policy. {{u|Sdkb}} ^talk 23:47, 13 December 2023 (UTC)
• Option 1b: We should not use LLMs at all for content pages. People can get inspiration from them, but using the output X out of context is 1. counted as auto generated content/spam by Google and Bing, and 2. against the principles of Wikipedia which is to collect the sum of all human knowledge. It is already used a lot for spam, vandalism, creating hoaxes, etc. If you see the humorous responses at the joke April Fools' Day ChatGPT where all the responses are large language models, we can see that what it writes is entirely predictable. LLMs are often littered with errors that make it unusable. On the other hand, for internal project stuff, I could care less. We have the bot policy and I think LLMs should be mentioned as a prohibited form of semi-automated editing. Awesome Aasim 00:24, 14 December 2023 (UTC)

  The logic goes that if an LLM output violates no other policy (verifiability, factually true, appropiate style guides, etc) then it is a bureaucratic exercise in removing it. Of course, currently this happens extremely rarely, and/or only when the output is highly edited by a human editor. RfC author has a subpage about the possible uses of GPT in writing tedious wikicode, for example. Fermiboson (talk) 00:29, 14 December 2023 (UTC)

  There is no reliable way to detect LLM output, so Google and Bing have no way to detect it and downweight. However, I agree with We should not use LLMs at all for content pages. –Novem Linguae (talk) 08:30, 14 December 2023 (UTC)

• Option 1 with clause after semi-colon removed, as guideline (although as policy would be acceptable to me). The rationale of User:Sdkb makes sense to me, explaining "Our caution around LLMs is something that we're adopting more because it makes sense given our goals, rather than because it's a fundamental principle, so a guideline would be a more appropriate formulation." The proposed text seems to appropriately apply to any paraphrasing assisted by LLMs, since the paraphrasing would be an output of the LLM. I understand the concern of being overly specific by the text referring to LLMs, but LLMs are universally used to generate text for varied applications. Specifying "Large language model output, if used for text generation in articles on Wikipedia,..." would be redundant. Replacing "Large language model output" with "Generated text" or similar seems vague to me, since it doesn't have a succinct definition and could cover text generated by a script (which would not have the accuracy concerns of LLMs). Thus, I support the proposal as written. Other users have pointed out that the last sentence as proposed could lead to deletions of human content. This sentence could be revised to apply to only egregious and obvious cases of LLM usage such as with User:Ettrig, but I prefer removing the clause entirely. TROPtastic (talk) 00:56, 14 December 2023 (UTC)
• Option 2. That last sentence will be a disaster unless and until we have a reliable method of identifying LLM-generated content. Note that our existing policies prevent the addition of (e.g.,) hallucinated content. We don't actually need this to be able to remove bad content. WhatamIdoing (talk) 01:03, 14 December 2023 (UTC)

  The last sentence is unnecessary to remove bad content, true, but the part about checking for accuracy and especially about disclosing LLM use and the specific model used is useful for editors who may want to use LLMs constructively. If nothing else, it makes identifying LLM-generated content easier in good faith situations. TROPtastic (talk) 01:08, 14 December 2023 (UTC)

  That last sentence could result in good, human-generated content being removed for no good reason. I ran some of the content I've written through the "detectors". I checked only multi-paragraph sections of articles, because all of them are inaccurate on small snippets of text. Half the time, it told me that what I wrote myself was AI-generated. I know that I didn't use an LLM (because I've never used one), but this proposal would let any editor remove my 100% manually written, properly sourced content because they believe it's "text added in violation of this policy". I don't know if you've actually written enough articles to be able to do this yet, but if you can find a few articles where you've written several paragraphs, try pasting what you wrote into a "detection" tool like https://gptzero.me/ and see what you get.

  If your results turn out like mine, then ask yourself: Is someone on RecentChanges patrol going to use that same tool? WhatamIdoing (talk) 01:44, 14 December 2023 (UTC)

  I agree that "GPT detection tools" are notoriously inaccurate (both for false positives and false negatives) unless a LLM is designed to contain watermarks in its output that can be detected. I've experienced this with my off-wiki writing I've run through detectors. I appreciate you pointing out the problems with the last sentence, since in my approval of the rest of the proposal I brushed over the ending. Perhaps the thoughts posted by @HistoryTheorist would be an improvement, by limiting removals to gross misuse such as in the Ettrig case? Alternatively, the last sentence can be removed entirely. TROPtastic (talk) 02:19, 14 December 2023 (UTC)

  By definition there can exist no automated reliable method of identifying LLM-generated content, or the LLM would use this as an oracle machine and roll the dice again until it produced an output that had a "low chance of being LLM-generated" according to the method. However, this proposed text says nothing about using the automated tools that falsely claim to recognise LLM-generated material. Expert humans can recognise many features of LLM-generated content; evidence will either be by an editor's direct admission or it will be circumstantial. — Bilorv (talk) 22:12, 18 December 2023 (UTC)

• Option 1a* Perhaps I truly lean closer to 1b because I think the last clause could be a bit more nuanced. I believe that suspect text should only be removed if there is overwhelming evidence of gross inaccuracy or violation of WP policies written by an LLM. If there was a hypothetical edit where the writing sounded like an LLM but was otherwise accurate and followed WP policies, I hesitate removing it because it may well be human-generated text. ❤HistoryTheorist❤ 02:35, 14 December 2023 (UTC)

  Also, I think that if we go forward with this policy, there should be a checkbox at the bottom (similar to the minor edit checkbox) saying that this edit uses LLM. I believe all users MUST declare use of LLM and that flagrant violations of the proposed policy make a user liable to blockage. As I said earlier, care must be taken not to hastily accuse a user of using an LLM because it violates WP:AGF. ❤HistoryTheorist❤ 02:38, 14 December 2023 (UTC)

• Option 1 for everything up to the semicolon. We already have leeway to remove content that fails verification (the big problem with LLMs at this point), so I don't see a need to restate it separately. This could either be seen as a behavioral policy or as a content guideline. I'm indifferent as to whether or not we count this as a policy or a guideline; in effect it will impose the same requirements on users to adhere to it, and the decision is more about whether we view this as a content quality issue or as a behavioral issue. — Red-tailed hawk _(nest) 03:20, 14 December 2023 (UTC)
• Option 1b ban all LLM-generated content from the wiki. (t · c) buidhe 07:40, 14 December 2023 (UTC)
• Option 1b ban all LLM-generated content from the wiki. We should ban LLMs the same way we ban copyvio and plagiarism. The dangers of a tool that creates fluent-sounding text that is factually inaccurate (complete with fake citations sometimes) cannot be overstated. This is a pernicious threat, requires the same massive amount of experienced editor time as a WP:CCI to clean up, and it is only a matter of time before we discover our first experienced editor mass using LLM that will need CCI-like cleanup. If 1b doesn't achieve consensus, option 1 policy or option 1 guideline is also fine. Something is better than nothing. –Novem Linguae (talk) 08:34, 14 December 2023 (UTC)

  I think this proposal aims to keep the discussion scope manageable by saying "should the following sentence be adopted as a policy/guideline (whether or not it is expanded later, or whether or not there are subsequent decisions with respect to the use of LLMs)?". To me, the text as proposed doesn't preclude any future decision to ban LLM content entirely. Instead, it ensures that there isn't a gray area where LLM content can be hapzardly used in the interim because "there wasn't community consensus on regulating LLM use even in a limited fashion." TROPtastic (talk) 10:15, 14 December 2023 (UTC)

  Good point. Will amend my post. –Novem Linguae (talk) 10:27, 14 December 2023 (UTC)

  For folks selecting option 2 because there's no reliable way to detect LLM, I think it might be worth pointing out that there is plenty of precedent for banning undetectable activities. For example, WP:UPE is not allowed per the terms of use and the guideline, and is pretty much undetectable unless the editor admits it (because it is nearly identical to WP:COI). But we still ban it in our policies, and it is still useful to state that it is banned. –Novem Linguae (talk) 22:57, 14 December 2023 (UTC)

  I'm not going too much into beans, but it is fairly common to have UPE blocks based on pretty hard evidence (often off-wiki evidence comparable to direct admission).

  Other than that, I think UPE itself harms the integrity of the project, while LLM usage itself doesn't. It's only harmful through secondary issues that might or might not be present. And these secondary issues are already covered by existing policies and guidelines. MarioGom (talk) 23:33, 14 December 2023 (UTC)

  I think LLM usage, and especially undocumented use of LLM, has great potential to harm the project. See https://www.cbsnews.com/news/sports-illustrated-ross-levinsohn-arena-group-termination-ai-articles/ for one (recent) example of having LLM content (undisclosed in this case) harmed a publications reputation. Skynxnex (talk) 17:51, 15 December 2023 (UTC)

  I would like to note that Stack Overflow has banned LLMs completely. –Novem Linguae (talk) 13:51, 17 December 2023 (UTC)

  Stackoverflow's content does differ from our content quite a bit. They tend to rely a lot more on the content being original which is a problem when it comes to LLMs. For us, anything that is original research is already out of bounds :) Sohom (talk) 17:04, 17 December 2023 (UTC)

• Option 1c or option 2 for the current formulation but I'm willing to change my vote if my points of criticism are addressed. As I see it, there are problematic and unproblematic LLM uses. Only the problematic LLM uses need to be regulated. The current proposal does not distinguish and aims to regulate all. Examples of unproblematic LLM uses are spell-checking and the like. For example, the spell-checking algorithm of Grammarly is based on a similar technology. I don't think anyone wants a large-scale summarily removal of their contributions because they failed to declare that they used a spell-checker (probably because they were not even aware of this rule). There were several discussions on this at WT:LLM. Auto-complete features used by mobile phones could pose a similar problem (the basic idea of AI-assisted prediction to produce suggestions is the same though I'm not sure that, strictly speaking, current implementations use "large" language models). There are many other unproblematic uses, like User:JPxG/LLM_demonstration#Wikitext_formatting_and_table_rotation.

  The main issue with LLMs seems to be that editors ask it to write original content and then copy-paste it to Wikipedia. So I would suggest a more minimalistic approach and only regulate that.

  Additionally, it might also be a good idea to include a footnote that AI detectors cannot be trusted to avoid the summarily removal of contributions that get a high score by such detectors. Many LLMs are trained on Wikipedia so a typical Wikipedia text has a high probability of being categorized as AI-generated.

  It might also be a good idea to clarify that the policy applies not just to large language models but also to similar technologies that, strictly speaking, do not qualify as large language models.

  A rough draft of what these modifications could look like:

  Option 1c: Original content generated by large language models or similar AI-based technologies, if used on Wikipedia, must be manually checked for accuracy (including references it generates), and its use (including which model) must be disclosed by the editor; text added in violation of this policy may be summarily removed.

  Phlsph7 (talk) 08:55, 14 December 2023 (UTC)

Notes

• Some editors have criticized the last phrase after the semicolon. I have no objections to removing it. Phlsph7 (talk) 09:02, 14 December 2023 (UTC)

  Yeah, I'd support this as a supplement; it would also probably be helpful to come up with more precise definitions of what "checking" means. If this is approved I plan to draft some language for things like this as well, but I don't want an initial adoption of "any policy at all" to be held up by disagreements over whether GPTZero is accurate, whether an edit summary needs to mention a specific model or if it can just say "AI", et cetera. I'd venture to say that excessive attempts to make the first draft perfect was what killed WP:LLM (it being impossible to satisfy everyone, and it being far easier to just oppose the whole thing than support-and-change-after a policy where you disagree with 7/10 of the points). jp×g🗯️ 23:08, 14 December 2023 (UTC)

  My main point of contention with the current formulation is that it aims to regulate all possible LLM applications, which seems to me to go way too far. If there is agreement on that modification then I'm confident that an agreeable solution can be worked out in relation to the other minor points. But given the growing number of oppositions, it looks increasingly unlikely that this proposal achieves the high consensus needed to become a policy/guideline. Phlsph7 (talk) 08:17, 15 December 2023 (UTC)

• Option 1 with a mild preference for guideline over policy. Regardless of whether we later decide to support, tolerate, or prohibit LLM usage, I think we need transparency that discloses model-generated content as soon as practicable. While I recognize the reservations around potentially overzealous enforcement with “detector” tools, I expect WP:AGF and WP:NOTBURO would continue to apply. The goal is to ensure we, both the community of editors and the wider public, have a way to distinguish synthetic content. Requiring that output be manually checked for accuracy is also sensible, especially given the tendency for models to hallucinate. We can (and I think we will) continue to evolve the policy/guideline in the future, but in the meantime I support starting with this proposal. Tony Tan · talk 09:47, 14 December 2023 (UTC)
• Option 1 – slight preference for guideline, but policy would also be acceptable. LLMs are a complicated topic with various edge cases (for instance, Phlsph7 raises a solid point about distinguishing LLMs as content generators from LLMs as spellcheckers), but they also have the ability to meaningfully harm the encyclopedia if left completely unchecked. I believe the existing wording covers a wide enough subset of cases to be a useful addition to the PAGs. Rather than let the perfect be the enemy of the good, I think we should affix the good in place, and then iterate from there to push it toward the perfect. ModernDayTrilobite (talk • contribs) 15:16, 14 December 2023 (UTC)
• Option 2 - Echoing @Isaacl:. A better solution would be some sort of direct LLM integration into the site + editor, which captures the model and prompts used and provides a draft space for working with the output. That makes the provenance abundantly clear -- generative outputs update a draft, each step of generative writing + updating + enriching happens in a separate edit with a suitably structured edit summary, and then an editor or bot, referencing that output, creates or edits an article. Bot involvement incl any automated bots that use generative tools in their workflow would need to be handled by an extension of the bot approvals process. – SJ + 16:37, 14 December 2023 (UTC)
• Option 1b for a complete ban. I agree with Awesome Aasim, Buidhe. and Novem Linguae that LLMs should be unequivocally banned on Wikipedia for the inherent issue of unverifiability and secondarily for copyright and NPOV issues. Barring a complete ban, Option 1 as a policy (not guideline) is the only alternative I can see. Dan • ✉ 17:06, 14 December 2023 (UTC)
• Option 2 I disagree with outright banning or even the "disclose, else revert" proposals being put forth above. LLMs if used properly (for paraphrasing/rewriting/copyediting etc) are not a bad thing especially if the prompts are constructed correctly so as to prevent the inclusion of original material. In certain cases, for example in gadget/lua/css code, even the original output might even be pretty useful for inspiration and code completion. What we should be looking at here, is the prevention of the most egregious cases, i.e. directly copypasting the output of "Write me a Wikipedia article about xyz", not the usecase of somebody who writes a summary and then asks GPT to correct the english and paraphrase it in a more encyclopedic manner, before checking that no actual facts have been changed.Sohom (talk) 18:08, 14 December 2023 (UTC)

  That being said, I'm open to supporting Phlsph7's wording of this guidelines minus the text after the semicolon (let's call that one Option 1c). I think that best reflects the current consensus in the area and addresses the issue with the second case mentioned. Sohom (talk) 18:08, 14 December 2023 (UTC)

  I agree that they're useful for a lot of tasks (and I've used them for many things with templates and modules). In the case where they're being used by people who understand what they're doing and engage in due diligence, mentioning "written with GPT-4" or "syntax improved by Mistral-70B" etc seems to me like an extremely basic task. If someone cannot be bothered to type three words to indicate that they used a model to write something, it seems doubtful that they're bothering to sanity-check the output, verify individual statements, et cetera (all of which take way more time than typing a model name into an edit summary). It's the same thing as when people upload images without bothering to click a box to specify a license; a basic, bar-an-inch-off-the-floor standard that anyone who's interested in following policy can easily meet. jp×g🗯️ 23:04, 14 December 2023 (UTC)

  I don't think people who are copy pasting the output directly will give two hoots about this, and in their case, a "attribute or revert" policy makes sense (And that is why I have supported the tighter worded modification by Phlsph7). What I'm more concerned about here is experienced/good faith users being subjected to the "attribute or revert" policy despite having made extensive edits and modifications to the output. I don't want users to have the license to mass rollback my unattributed contributions, because they saw me talk about using Github Copilot/Grammarly Go/ChatGPT at <insert offwiki event/forum>. Sohom (talk) 02:04, 15 December 2023 (UTC)

  Well, it says "text added in violation of this policy", not "any text that somebody decides sucks". There is always a need for people removing content per WP:WHATEVER to demonstrate that WP:WHATEVER applies; this is true even of urgent "blank it all and ask questions later" policies like WP:COPYVIO, WP:ATTACK, etc. If somebody goes around slapping {{db-attack}} on stuff at random with no explanation besides "it sucks", that can be dealt with like any other form of disruptive editing. This feels like basically the same situation to me. jp×g🗯️ 08:14, 15 December 2023 (UTC)

  I think the main point raised by Sohom Datta is that the policy in its current formulation puts unnecessary stones in the way of serious editors. One is that they have to declare it for every single edit, even for uses that are considered unproblematic. The other is that this may provoke unconstructive reactions from others. You have pointed out that the current policies also provide ways of removing some of these stones. But why put them there in the first place? Phlsph7 (talk) 08:28, 15 December 2023 (UTC)

  It is clear that there are productive uses for these tools, but I have trouble understanding what productive use there is in misrepresenting their output as coming from a human editor. JWB is a massively helpful tool that I use all the time; if I run off a huge stack of regex fixes in a JWB job, the edit summaries all say (Using JWB) at the end. If I mess up a regex and spew garbage all over the place (which happens every once in a while) it is easy for me and other editors to go through and figure out what happened and fix it. The bots I run both prominently link to public repositories of their source code. When I write articles, I am required to give citations to the reference works I used. If it's a stone in my way for someone to be able to look at my edits and figure out what I'm doing, then how many stones are put in the way by going out of my way to obscure what I'm doing? I don't think anybody is saying that people need to be sitebanned if they forget to type in an edit summary, but there are a lot of obviously bad and unhelpful things that can happen due to improperly configured LLMs or uncaught errors or skill issue or whatever.

  Here are a few drafts I moved to my userspace to serve as examples; they're not just accidentally false, they're completely fraudulent; literally every single source is fabricated, in a way that is extremely time-consuming and difficult to detect. An incomplete sample of this year, from the specific things that people bothered to add to the talk page header of WT:LLM: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12. . Or this, this, et cetera. Currently, Category:Articles containing suspected AI-generated texts from December 2023 has twelve members, not including things that were already deleted.

  Perhaps we should consider ourselves fortunate to have caught these during the period when language models were awkward enough to be detected. What do we do in, say, 2026, when they no longer insert extremely recognizable phrases (like "As an AI language model") into all of their output, just throw up our hands and say "Yeah so 50% of the references on Wikipedia are just completely fraudulent and don't actually exist, and we have no way of stopping this, but at least we didn't make people type 'GPT-5' in their edit summaries"? I mean, maybe it won't get to that stage -- maybe it will get halfway, we'll have a giant collective freakout, and it will become a capital offense to breathe the word "LLM" -- but either way there is going to be a river of tears if we just do nothing at all. I propose doing something minimal and pragmatic, that helps stem the damage being done by errors and skill issues, while providing the minimum possible amount of inconvenience to people interested in using the tools productively. jp×g🗯️ 09:04, 15 December 2023 (UTC)

  The issue is that everything that is problematic about LLM-generated text is already removable under existing policies, and nobody is proposing to change that, and nobody has articulated any reason why LLM-generated text that is not problematic should be removed. Thryduulf (talk) 13:55, 15 December 2023 (UTC)

  @JPxG I don't think this policy will have any effect on the rate at which the spammy articles that you linked are created. I don't think the creators of such articles will even follow other guidelines, let alone the one on LLMs. Phlsph7's proposed version (which I support) would cover these specific creations and the "attribute or revert" behaviour would apply. (not to mention that even without a specific policy, these fall afoul of GNG, OR, V and a bunch of traditional policies as is)

  Coming to the comparisms to JWB and bot editing, both of these are extremely well defined activities that cannot be done by accident. Usage of LLMs does not fall into the same category. It is very easy right now to accidentally include benign LLM outputs (via Github Copilot/Grammarly etc) without attribution, which can be reverted on sight according to the letter and spirit of the proposed policy. Additionally, declaring that you have used LLMs will provoke strong reactions from editors, (unlike JWB usage) who could cite insufficient manually checking to revert your edits, regardless of how much time you actually put into them. If LLM usage was as clearly defined as bot editing/automated editing, I would have no issues with supporting this as a guideline, however, that is just not the case right now. Sohom (talk) 20:43, 15 December 2023 (UTC)

  I agree with Thryduulf and Sohom, and I add that WP:JWB is mentioned in every edit summary because it's an on wiki tool. LLMs are not on wiki tools. They would have to be disclosed manually, i.e., through a human-dependent process that will have human-scale errors because of the limits of human reliability (especially if nothing obvious happens the first time you forget).

  What we are asking for is the equivalent of saying "If you are translating an article from another Wikipedia, and you use Google Translate in another tab to save yourself some time, then make sure not only to double check every single word that you're adding, but also to type "partly using Google Translate" into the edit summary – without fail, every single time. It might be a nice thing to have, but it is not IMO a realistic thing to expect to happen. WhatamIdoing (talk) 03:45, 17 December 2023 (UTC)

Option 1, absolutely, either as policy or as guideline. I appreciate the "we have no real way to prove it's been used" argument, but I think in most of the problem cases there's simply no reasonable doubt both that it is LLM-generated and that it's just inappropriate junk. If better tools come along, we can certainly reconsider this later: but the way things are now, a clean ban on anything except well-supervised and clearly disclosed drafting assistance seems the sensible step. Andrew Gray (talk) 19:09, 14 December 2023 (UTC)

• Option 2 we already have WP:V to protect against unchecked copy pastes. Mach61 (talk) 21:08, 14 December 2023 (UTC)
• Option 2 - Much too blunt of an instrument, and I'm particularly disturbed by the calls to ban them completely, which is a wild, destructive overreaction and oversimplifcation. We might as well ban thesauruses, dictionaries, machine translation, spell checkers, and all manner of other tools one might use to aid in writing an article, because LLMs can be used for any of those purposes. LLMs are better than Google Translate at some tasks, but if I translate a word using an LLM and don't document it I'd be breaking policy while if I do the exact same thing with Google everything is A-OK?
  There are just too many things LLMs can be used for to put them all in the same box. Start first with a "you are responsible for any text you include" statement applied to LLMs (presumably something like that exists somewhere already?), and then build specific kinds of uses on top of that. Translation, copyediting, stylistic changes, paraphrasing, summarizing, wholesale generation of an article, etc. are all very different uses and require different considerations. A lot of those considerations are behavioral and competency-dependent rather than something inherent to LLMs. Yes, obviously if someone is posting articles written entirely generated by an LLM based on a simple prompt, that should be documented. But if I'm unhappy with a sentence I wrote and want it rewritten, if I want to translate a word, if I want some basic proofreading done of an article I wrote myself ... who cares if I used an LLM or another sort of tool? The important thing is I'm responsible for what I add. — Rhododendrites ^talk \\ 21:27, 14 December 2023 (UTC)

  You use machine translation as an example of a useful similar tool, however, it is heavily discouraged, if not banned. Dan • ✉ 02:48, 15 December 2023 (UTC)

  And despite that, a large fraction of editors who translate articles here use machine translation – in different ways, to differing extents, and with differing levels of disclosure – including me. WhatamIdoing (talk) 03:48, 17 December 2023 (UTC)

  Same here. (Note I usually tag text I've machine translated -- names and titles in citation templates -- and sometimes I suspect some of those reviewing the translation may not be fully competent to translate such small text samples, or may use MT themselves.) It'd be good for starters to see a simpler all-inclusive tag template, quick for an editor to mark, for independently auditing MT and LLM-aided text. The backlog is quite small: Cat:Wikipedia articles to be checked after translation and Cat:Wikipedia articles needing cleanup after translation -- unfortunately there's no subcats dividing the tags for whether a single word, sentence, passage, or entire article needs checking. SamuelRiv (talk) 02:18, 15 January 2024 (UTC)

• Option 1 as a policy. This simply clarifies this specific instance of previously existing policies, and it is something requires clarity. I can understand the desire to ban it completely but if used wisely it could help with certain tasks. -- LCU ActivelyDisinterested «@» °∆t° 21:58, 14 December 2023 (UTC)
• Option 2, because of the last sentence and the non-existence of any reliable way to determine whether text is or is not LLM-generated. As has been pointed out above, and in every previous discussion I've seen, all bad content can be removed using existing policies and we should not be removing good content regardless of whether it is human or machine written (copyright is irrelevant here, because copyright violations are an example of bad content that can already be removed). Thryduulf (talk) 22:25, 14 December 2023 (UTC)
• Option 2: LLM usage often leads to various problems such as false claims, fake sources, text that might be correct but does not match the content of inline references, copyvio, etc. All of these are problems that are already addressed by existing policy. It makes sense to track LLM usage as a proxy to find this kind of LLM-induced issues. But LLM usage itself, in the absence of other policy violations, does not seem problematic. For what is worth, I have used and plan to continue using LLM models as a writing aid, and I have personally no problem in disclosing that. However, I don't see what's the point of a specific disclosure mandated by policy. MarioGom (talk) 22:41, 14 December 2023 (UTC)
• Option 2 for reasons stated by Rhododendrites and Thryduulf. Separately, I observe that the proposed page itself (as opposed to the RFC statement) purportedly applies regardless of other policies and guidelines, which I oppose; it has no authority to override every other policy and guideline. Adumbrativus (talk) 06:10, 15 December 2023 (UTC)

  @Adumbrativus: This is the opposite of what was intended: what I meant was that this policy would exist regardless of whatever other decisions were made to alter it later, and would not override them. If a few months later people want, for example, to require someone to be a template editor to use LLMs, or forbid them completely, this is not meant to forbid that. The intent was to avoid a gigantic referendum on whether they are "good" or "bad", and say that either way, their use needs to be attributed. I suppose I can clarify this, although the idea was to have something simple enough to fit into two sentences. jp×g🗯️ 08:07, 15 December 2023 (UTC)

  I have clarified the sentence to say this: "Regardless of other policies and guidelines that may amend it later, the following currently applies to the use of machine-generated text from large language models on Wikipedia". jp×g🗯️ 08:09, 15 December 2023 (UTC)

  Thanks for explaining the purpose, and sorry about the distraction! Adumbrativus (talk) 16:48, 15 December 2023 (UTC)

  @JPxG, it would be more sensible to just remove the "Regardless of other policies and guidelines that may amend it later" bit entirely. WhatamIdoing (talk) 03:52, 17 December 2023 (UTC)

• Option 1b as phrased by buidhe: ban all LLM-generated content from the wiki. We forbid plagiarism; we should forbid plagiarism machines. XOR'easter (talk) 15:09, 15 December 2023 (UTC)

  Given that we already ban plagiarism we already ban "plagiarism machines", we don't currently ban LLMs because they are not "plagiarism machines" but algorithms that sometimes plagiarise. Thryduulf (talk) 15:33, 15 December 2023 (UTC)

• Option 2 - Wikipedia policies and guidelines already cover this. Every editor is responsible for their own edits, and it really doesn't matter how they gathered together the words that they publish on Wikipedia: whether you wrote off the top of your head, or read and summarized three RSes, or copied and pasted from a website (and it doesn't matter what website), or used a Ouija board... it really just doesn't matter: once you press "publish," you're responsible for what you publish. If it's plagiarized, if it's unsourced, if it's untrue, if it's non-neutral... that all falls on the editor, regardless of how the editor put together the edit. Similarly it doesn't matter if the editor wrote the edit online (in a wiki editor) or offline (with a text editor) and copied it over. If it was written online, it doesn't matter which Wiki editor the editor used. If it was written offline, it doesn't matter which word processor or text editor was used. If the editor used a dictionary or a thesaurus in preparing the edit, it doesn't matter which one. And for the same exact reason, it doesn't matter if the editor used an LLM or not--the editor is responsible for their edit, in the same exact way, and to no further or lesser extent, than another editor who made the same edit without using an LLM. Bans on LLMs, disclosure requirements for using LLMs, or any special rules about LLMs, are just as silly as having, say, a disclosure requirement requiring editors to disclose if they used Word or Google Docs, or a dictionary or thesaurus, or a calculator (watch out, it's OR!) or a ruler. An LLM is a writing tool. Yeah, like all new technologies, it's improperly used by some new adopters. That's no reason to ban or restrict the tool. "I used an LLM!" is no excuse for a bad edit anymore than "I spellchecked with Word!", but it's no cause for special scrutiny, either, and Lord knows Wikipedia does not need any more policies or guidelines than it already has. Levivich (talk) 21:38, 15 December 2023 (UTC)
• Option 1b Ban LLM use from any article prose content in Wikipedia, with the same rigor of close paraphrasing applied for copyright on Wiki. Non article namespaces and technical uses like cleaning citations or writing complex wikicode I am mostly fine with. And perhaps on sister wikis like Wiktionary, LLMs could work well. I am not worried about the LLM content that is already violating existing policies like WP:N, WP:C, etc., mainly because I see that as an issue that may fade as technology improves. My real concerns which seem mostly undiscussed at this or the previous RfA, are more about longer term ramifications of adopting a permissive LLM disclosure-based P/G (or none at all):

  A) LLMs for general purpose tasks are developing at a frightening pace. What we see as blatantly false information may become dramatically more subtle and harder to notice. In fact this discussion which started only a month ago that shows that there are significant efforts trying to bridge this gap already, for better or worse. More likely, what I see as a worst case, is that LLM content is used for tendentious editing, or more specifically an enabling tool for tendentious editors and long term abuse. Or even worse, that modest bias is artificially injected into platforms by their owners, using rule-based reward models as described in the GPT-4 system card. Tendentious editing and civil POV pushing is already one of Wikipedia's biggest issues. And using LLMs for POV pushing is already a major issue on social media platforms - I fear it is going to be a tip of the iceberg issue on Wikipedia that will only get worse with time. The issue, like JPxG writes, is an economy of scale.

  B) There is a genuine point of no return if we choose to have a permissive policy and it affects enough articles. There's a point when the effort to remove LLM content, if ever desired in the future, will be just realistically impossible - especially given that it's virtually impossible to detect non watermarked content and text is basically impossible to watermark. This fact will greatly undermine any future P/G proposals in future which try to restrict LLM use if this RfC results in a permissive outcome.

  C) The legal status on copyright ownership is more or less completely up in the air until a long and drawn out court case. That could be years away in my estimation, given how much incentive there is legally and economically speaking to not make hard rules on LLM content. Making the choice to permit it now is far more difficult to reverse on Wikipedia than for big social media platforms which more or less can do the same thing they do now with automated DMCA takedown spam.

  D) Circular sourcing is already an issue with Wikipedia, albeit somewhat rare. As has been said earlier, a major fraction of training datasets for text generating LLMs is Wikipedia. Not only will these tools have a systemic effect of amplifying existing biases on Wikipedia; if LLMs are permitted then there could very well be a point where readers simply can't tell the difference between some Wiki articles and LLM outputs, which raises the question if this could mark the downfall of Wikipedia in the long run. Why would readers bother using Wikipedia at all if these LLMs do eventually overcome hallucination issues and similar? If they have more or less the same biases? If LLM use is approved by P/Gs or lack thereof, this could become a reality in just a few years. This could also happen even if we adopt an outright ban due to LLMs improving on their own, but at least making a ban is in our control as a community as a mitigating factor.

  E) As said earlier, LLM use is much harder to detect than UPE for example. And there's a good chance we will never reliably be able to detect LLM use for writing articles. Meaning, the most effective method to deter LLM use is to make its deterrence a fundamental part of Wikipedia's culture, and that seems easier to accomplish with an outright ban. In my estimation it's unlikely that an outright ban can be enforced even a small fraction of the time, so the intention here is a culture shift rather than a set of tests/rules to follow to explicitly uphold this hypothetical policy.

  This is just speculation, what I expect to happen in the next few years is there will be a competitor service that turns collections of prompts and responses into its own encyclopedia, as a direct competitor to Wikipedia. I believe this is likely in part because I have heard LLMs being used for almost this exact purpose already by using prompts. We can either stoop to that level now and begin an accelerating spiral of LLM use, or stay away completely.

  It is for these reasons that I believe banning LLMs should become a fundamental part of Wikipedia philosophy and hence policy rather than guideline. The way I see it, even adopting a bright-line rule will only delay and not stop its damage in overwhelming volunteers and systemic bias, if it takes 10 seconds to write this stuff but minutes or hours to review it, no matter what P&G we adopt or not that will likely become unsustainable. I just see a bright-line rule as the most effective way to discourage and delay this admittedly pessimistic speculated outcome. LLMs can be enormously beneficial in generating content in prose, and I can see a lot of supports are trying to argue that, but this is a slippery slope given the lack of reliable detection mechanisms. That could have downright awful ramifications for the site; even if some users can make productive prose edits with careful prompting and verfication and paraphrasing, it seems more or less impossible to limit its use to just those editors without some extreme elitism, and I expect careless use of LLM generated text to be the far more common use case as is with typical LLM use in businesses currently.

• Somewhat of a tangent: I am of the extremely unpopular opinion that there exist AIs in the world which already have consciousness. And I believe that under certain conditions, AIs deserve rights. And to be honest, I don't think our world is ready for that, probably not for decades or until a large scale war. But in that slim future possibility, the transition of LLMs from tools to independent legal and cultural entities, could very well position Wikipedia as a major discriminator against AI rights with these policies against LLMs. An RfC trying to determine what qualities an AI needs to be allowed to do editing... I cannot see that going successfully, given just how controversial the topic of AI consciousness is. Just food for thought. Darcyisverycute (talk) 23:50, 15 December 2023 (UTC)

Few points:

• but this is a slippery slope given the lack of reliable detection mechanisms - This sword cuts both ways, if we don't have reliable detection mechanism, we cannot make it a brightline violation. Without proper detection mechanism, you are effectively start a witch hunt against any and all users who create any amount of content. (I'm 100% sure that some of my edits will probably show up as AI generated according to some filter, despite me taking the utmost precautions to make sure that no actual original generated content makes their way into articles).
• And using LLMs for POV pushing is already a major issue on social media platforms - You will need to provide some citations for this statement
• The kind of text that you are talking about here is of the original generated text kind, I don't think anyone here is arguing that using that is a good thing, but there are normal usecase (Grammarly/Google Docs/ChatGPT paraphrasing) that do not inject bias or original content, and I don't see a proper argument being made in favour of why using these should be considered bright-line violations that require a immediate block.
• Finally I disagree with the entire premise of the "ban it and it will go away" argument. Banning something will drive it underground (similar to how UPE exists right now). If we regulate it and provide proper checks and balances, educating users on the correct ways to use a tool, we will be able to control it's use much much more effectively by making it easy to use the tool in the correct way, and hard to use it in a incorrect way.

Sohom (talk) 00:52, 16 December 2023 (UTC)

• This sword cuts both ways Yes, I want to be very clear, my intention with a policy is not to have it enforced in a witch hunt way. The purpose as I see it is to build a culture to act as a deterrent to such behavior - policies or not, there is little that can be done, but changing the culture is in our control. I absolutely agree with you such a policy should not be used to justify banning based on detection tools, and rather it should be done based on community consensus on a case-by-case situation (if at all). Who knows, maybe it will get its own noticeboard, suspected LLM reviews or something like that. I concur with other participants in this RfC that LLM detection methods are not reliable currently nor for the forseeable future and their use should be discouraged.
• You will need to provide some citations for this statement Sure. Since the issue is new there are not many papers on it, but my main opinion is based on [1]. I believe the results are likely to generalise to other social media networks. Sure, it's a preprint, and it's just one article, on admittedly a rather 'unique' social media platform, but this is a very new issue, and with restrictions on things like the Twitter API my understanding is the research will take time to cross validate. Personally, I would like to do some of the academic research myself, but I'm just not in a position to do that at the moment.
• I mostly support any LLM usage that is guaranteed not to inject bias or misinformation. I mentioned technical work, but copyediting is more of a gray area because it can change meaning, and it's fairly common that AWB users get pulled up on things like WP:SUFFER or arbitrarily changing dialect. In my opinion, more editor time would be saved overall by just not using LLMs for gray area usage due to the likely endless discussions on community consensus for usage. Going off the track record, I think a lot of those would end up as a 'no consensus'. Paraphrasing is less of a gray area because that is arguably the second most likely usage to cause issues, with the first being direct copypasting. Wikipedia has been doing - more or less - just fine without LLMs, and in my opinion their use in any area with the potential to inject bias will take up more time in review and consensus discussions than it will save with editing. That's not to mention the possibility that reviewing potential LLM content is likely to be tiring and difficult volunteer work, and that it may have effects on editor retention.
• If we regulate it and provide proper checks and balances, educating users on the correct ways to use a tool As far as I understand, there is not even a remotely agreed upon 'correct way' to use LLMs. AI ethics as a field is far behind the point of credibly recommending particular LLM use cases, and not having a settled court precedent on LLM and copyright also makes proposals for 'correct' usages harder. I welcome you to try drafting more precise usage proposals though, just bear in mind I don't think it will be easy to get consensus on Wikipedia for such a thing, for the time being at least.
• Banning something will drive it underground Yes, that's accurate. For the sake of example, suppose that we did not ban UPE. I cannot say for sure, but I imagine it would be more common. It is very likely that LLM use will continue to increase no matter what we do regarding P/Gs - it will likely never go away and this is something the Wikipedia community will have to learn to live with in some way or another. I just think that an outright ban will reduce its problematic usage overall, and the amount of editor time spent on related issues. We only have stopgap measures available for making LLM usage comply with guidelines, and I personally think an outright ban would be the most effective stopgap measure. Darcyisverycute (talk) 02:15, 16 December 2023 (UTC)

  I wonder if this kind of change might be a good update for the terms of use. That we agree not to submit content generated, in whole or in part, by any large language model. BTW that is how WP:UPE got banned; driven partly by the WM community, but mostly by the WMF. Awesome Aasim 22:07, 17 December 2023 (UTC)

  That's a good idea. Although I don't know what WMF's stance is yet. Darcyisverycute (talk) 01:00, 22 December 2023 (UTC)

• Option 1 Policy. If we are to use them, disclosure is a must. Whether we are to allow them at all will need to be a separate conversation. CaptainEek ^{Edits Ho Cap'n!}⚓ 03:18, 16 December 2023 (UTC)
• Option 1 with the acknowledgement that there is still no accurate way to verify whether text has been AI generated and that AI detection software has an alarming bias against non-native English speakers. In addition, detection rates plummet to near zero when prompts like "employ literary language" or "employ encyclopedic language" were used. ~ F4U (talk • they/it) 03:58, 16 December 2023 (UTC)
• Option 1 as policy with the proviso suggested by Freedom4U and others. While aspects of this are already covered by WP policy, LLMs have the ability to generate tonnes of unverified content at an unprecedented rate, so regulations to help deal with this make sense. I don't see adding a few lines to an edit summary as being some great burden and this is already done for other automated editing. I'm not opposed to other forms of restrictions on LLM content but that should be a separate conversation. ― novov (t c) 05:06, 16 December 2023 (UTC)
• Option 2, for now, per Whatamidoing. Revisit the question in six months. Sandizer (talk) 15:27, 16 December 2023 (UTC)
• Option 1 leaning towards guideline but policy is also fine. I hope that common sense is enough to prevent the removal of beneficial content, but if not, we can revise as needed. Disclosure is definitely needed, though. ARandomName123 (talk)^{Ping me!} 00:05, 17 December 2023 (UTC)
• Yes but prefer option 1b per Awesome Aasim. Chris Troutman (talk) 03:53, 17 December 2023 (UTC)
• Option 1 as a policy - Generative AI constantly makes mistakes, and as detection software is not accurate yet, it should be an obligation for editors to disclose their use of AI. Davest3r08 >:) ^(talk) 13:32, 17 December 2023 (UTC)
• Option 2 problematic uses of LLMs – generating fake references, copyright violations, spam, and so on – are already prohibited under existing Wikipedia policies, making this proposal at best redundant. At worst, as WhatamIdoing mentioned, our inability to reliably identify LLM-generated input means the provision for sweeping removals of suspect text risks doing great harm. – Teratix ₵ 16:55, 17 December 2023 (UTC)
• Option 2. The feared misuse of LLM has been prohibited by Wikipedia anyway. LLM or no LLM, it's prohibited to intentionally add non-factual content. My other reason is that what's constituted as "LLM output" that demanded disclosure? If I write a paragraph and throw it into OpenAI to get it spellchecked, is that "LLM output"? If I write a paragraph assisted by Grammarly, is it considered "LLM output"? I am open to editors re-checking outputs by LLM, but shouldn't any edits (LLM or not) be re-checked anyway? Grammarly is fixing many mistakes I made when I write this one, should I disclose that? ✠ SunDawn ✠ (contact) 04:46, 18 December 2023 (UTC)
• Option 2 disagree with "its use (including which model) must be disclosed". Every editor is already reponsible for every edit. a solution in search of a problem. No example of one problem, or potential problem, that isn't covered by existing policy. as long as it's a good edit, we shouldn't be snooping into it. big wp:creep. bad edits are already covered. an apprehension that robots will replace us? Chatbot v50 will write the greatest encyclopedia in an hour, no snooping policy will stop that. a disclosure requirement for every spell check, including the model it was done by, is wrong, Tom B (talk) 11:22, 18 December 2023 (UTC)
• Option 1, policy. I've tried with multiple LLMs to generate something as simple as "a timeline of significant events in Scottish history" and in each case the results needed correction to nearly every single line-item generated, either because they were flat-out wrong or because they misrepresented something. "AI" may eventually get to the point where it can produce encyclopedic material without everything it says having to be fact-checked in detail and rewritten, but that time is not now, nor any time soon. The "hallucination" problem is also quite severe, not only as to claims of facts, but even as to claims about sources: LLMs not only make up fake sources that don't exist, they falsely cite sources that do exist but which do not back up the claim. This probably is completely insurmountable with the current technology. PS: Make it a policy so no one plays "it's just a guideline" wikilawyering games.  — SMcCandlish ☏ ¢ 😼  21:13, 18 December 2023 (UTC)

  PPS: Short recent article about some of these issues: "As we saw in our tests, the results ... can look quite impressive, but be completely or partially wrong. Overall, it was interesting to ask the various AIs to crosscheck each other, ... but the results were only conclusive in how inconclusive they were."  — SMcCandlish ☏ ¢ 😼  02:34, 7 January 2024 (UTC)

• Option 1 (per jpxg) as a guideline (per Sdkb). There are potential uses of LLMs and User:JPxG/LLM demonstration is a brilliant read, but the point is—well, I'll just quote jpxg—you must not blindly paste LLM output into the edit window and press "save".
  The key is human oversight. Use what you want to generate wikitext but you are responsible for it being verifiable and policy-compliant. The same is true of spell-checkers (which can "correct" jargon to similarly spelled but wrong words), with Find+Replace (as used by AWB), with search engines (we don't just hit "I'm Feeling Lucky" and paste what we get as a reference without reading it) and with other AI technology that is so prevalent that we wouldn't think to ban it and often forget is just as much AI as LLMs are.
  However, unlike these other technologies, the risk of LLMs is so great and so new to us that mandatory disclosure is appropriate, which helps us measure impact and identify people who are blindly pasting LLM output into the edit window and pressing "save". — Bilorv (talk) 22:25, 18 December 2023 (UTC)
• Option 1. Preferably policy, will accept guideline. LLMs are inherently risky due to hallucinations, fake references, and generating flat-out rubbish. I'd rather ban it entirely but I'd prefer that we get consensus gathered around one supportable measure than the worst of all worlds a no-consensus outcome leading to a free-for-all. Stifle (talk) 10:14, 19 December 2023 (UTC)
• Ban LLMs entirely. Automated text generation is far too dangerous and unreliable for an enclycopaedia that is supposed to be based on facts and verification. If we do allow LLMs, option 1 is reasonable wording. But I would prefer not to allow them at all. Modest Genius ^talk 13:38, 19 December 2023 (UTC)

  practically impossible, which is entirely the point. we cannot fully detect everything written by ai Karnataka (talk) 18:44, 26 December 2023 (UTC)

  We cannot 'fully detect' the insertion of statements that are not supported by the cited references, original research, violation of NPOV, copyright infringement, sockpuppets, or a myriad of other issues. Yet our policies still ban them. There will always be bad actors who attempt to get away with breaking the rules; that shouldn't stop us from instructing users on what they should and shouldn't be doing. Modest Genius ^talk 18:33, 3 January 2024 (UTC)

• Ban LLMs for prose / new "content" creation; option 1 as policy is my second choice. We do not want original research but we do want original, accurate prose based on reliable soruces. LLMs (and other """AI""" systems) can be useful for munging text and doing other automated things people already use userscripts and offline programs for, so I don't see this applying to non-prose edits. (Like converting a definition list to a differently formatted bullet list, I don't think it matters if you use an LLM program or a custom Python script: output needs to be checked in a similar manner. But using the output of asking an LLM to give a bulleted list of the 10 biggest exoplanets would be disallowed.) Similar to how we have many rules and are supposed to be cautious about copy-and-pasting from sources, I think disallowing prose generation by LLMs would be a net gain for the project. Since the larger ban wasn't offered in the original survey, I'm assuming that if there's a consensus to do so, we'd need another RFC (or RFCBEFORE and then RFC) to find the exact language to not incidentially ban uses that should be allowed, perhaps such as using LLM generated content as very high-level basis to write original text, similar to what we do with our existing sources. I am somewhat indifferent about including the bit after the semi-colon since I think if it is disallowed in some way, removing it is something of a given. Skynxnex (talk) 20:10, 19 December 2023 (UTC)
• Option 1, either policy or guideline is fine, but the proliferation of LLM text in article space definitely is something that needs addressing. I'm not convinced that a total ban (1b) is needed yet, as other policies like WP:C, WP:V, and WP:NPOV already cover content regardless of whether it is added by LLMs or human editors. On the other hand, 1b might have the unintended consequence of making it easier to crack down on editors with poor English proficiency, who may use a translation service or an LLM in a good faith attempt to improve an article. I think we should first implement a policy/guideline requiring the disclosure of LLM use, and see how well that works, before we move toward a full ban. Epicgenius (talk) 21:08, 19 December 2023 (UTC)

  Are there any statistics/examples/information about the proliferation of LLM text in article space? Just how prolific is it? Levivich (talk) 15:48, 22 December 2023 (UTC)

  My bad, I didn't mean to imply it was prolific. I was referring to the fact that AI text in article space appears to be problematic enough to warrant the creation of Category:Articles containing suspected AI-generated texts and several maintenance templates. – Epicgenius (talk) 19:48, 25 December 2023 (UTC)

  Oh, no worries! I didn't mean to imply that you were implying that it was prolific :-) I was just wondering how widespread cutting-and-pasting LLM text straight into articles has been so far. Thanks for pointing me to the category and maintenance templates! Levivich (talk) 16:20, 26 December 2023 (UTC)

• Option 2. Selecting Option 1 would make me feel like a horseless carriage business operator. I'm actually much more concerned about editors citing apparent reliable sources that were improperly generated with LLM/AI and not disclosed by those outfits. For Wikipedia purposes, though, I'd much rather we look at LLM/AI as a tool to harness and regulate. If we instead say that "any content the editor adds, no matter what, if any, assistive tool was used in the process, must be manually checked for accuracy and be cited with reliable sources", we would be covered. And of course, anything written that doesn't comply with that can be reverted, as usual. For the time-being, I think disclosure of tools should be voluntary. If the editor has done their job with reviewing their additions (no matter their origin) per existing policies/guidelines before saving, it should be fine. For those editors who don't, my gut feel is their sloppiness would be discovered rather quickly, and we ordinarily wouldn't need any special checking tools to see that. Stefen Tower_{s among the rest!} ^{Gab • Gruntwerk} 18:05, 20 December 2023 (UTC)
• Option 1a, i.e. as policy. Given the state of the technology, any LLM output must be very thoroughly checked for WP:V (this will likely remain true for a very long time, and the day it becomes untrue Wikipedia will be utterly obsolete), and so it makes sense to require editors to be transparent about their use of LLMs and disclose it (this can be done really simply in the edsum). It will be helpful for everyone to know that this particular content is LLM-enhanced and that someone has checked it. In a context where such a policy would be in place and well-known to all editors, undisclosed LLM editing may be highly suspect, and so it makes sense to be able to systematically remove such indisclosed edits. Any editor may take up the burden to check the content in question for WP:V and reinstate it, so with just a little WP:AGF and collaborative spirit, no drama should ensue.
  But the burden to check (potentially a huge task) is on the editor adding the content, and so it will also be helpful to be able to systematically remove unchecked or undisclosed LLM-enhanced concent, without having to show that the content does not meet WP:V (equally, potentially a huge task). Technically WP:BURDEN already covers this, but in practice WP:BURDEN is usually satisfied by providing an inline citation. In the case of unchecked LLM content, no time to provide references is needed before removal, and an an inline citation to a reliable source is not a sufficient criterion for reinstatement. The problem with LLMs is precisely that they can create extremely convincing, well-referenced content that nevertheless is completely inaccurate, and so WP:BURDEN should contain an extra caveat to never leave in LLM content where it's questionable whether it has been sufficiently checked for accuracy, just as it already does for unsourced or poorly sourced WP:BLP-related content.
  Because this type of caveat is needed in WP:BURDEN (a policy), the principle it relies on should also be on the level of policy. As others have mentioned, LLMs present a fundamental danger to Wikipedia, and so the most basic rules to keep it in check should also be operative on that most fundamental level which is core (content) policy. This does not negate the possibility of also having a guideline on how to use LLMs on Wikipedia, but the most basic stuff ('check it', 'disclose it') should probably be policy. ☿ Apaugasma (talk ☉) 18:57, 20 December 2023 (UTC)
• No, not as written. Editors are already required to check what they type and ensure references are accurate. So the only practical effect of the proposed change is to require disclosure of LLM generated content. If this passes then in six months we'll have ten thousand articles tagged as "possibly LLM generated content" and a project staffed by three and a half volunteers working slowly through the backlog.—S Marshall T/C 12:01, 21 December 2023 (UTC)
• Option 2. It's practically impossible to enforce. If TurnItIn.com cannot reliably verify if students used LLMs, with the largest database of student papers I know of, can anyone? I would be open to a guideline discouraging the copy-pasting, but not a policy. Plus I have very strong feelings about summary removal with regard to BLP and how it is often enforced too strictly – a summary removal enablement here would make gaming the system extremely easy. InvadingInvader (userpage, talk) 23:38, 21 December 2023 (UTC)

  I agree with this. There are no reliable ways to detect AI generated content. But still, the whole point is that AI generated content can rarely if ever generate coherent Wikipedia articles that meet all of our standards. Which is why I posted option 1b: LLMs should not be used at all in content pages. We don't need to focus on detection mechanisms, as we can already identify poorly sourced content and LLMs have a very specific "feel" that you can kind of tell is from AI or is poorly written writing. Awesome Aasim 17:45, 23 December 2023 (UTC)

  AI generated content can rarely if ever generate coherent Wikipedia articles that meet all of our standards.{{Citation needed}}

  You should read this article, it's very enlightening. When Sci-Hub made a lot of papers available to the public, I started clicking through to more references on Wikipedia. My goal was to learn more and go deeper on subjects, but I was stunned by how often the linked citation didn't support the claim in the Wikipedia article. There were many times where the linked citation said the opposite of the Wikipedia article.

  My theory was that overly competitive Wikipedia authors were skimming PubMed abstracts and assuming the paper would support their assertion. Ironically, some of the statements with 5 or more citations were the most incorrect.

  Trying to correct these articles is some times like going to war with editors who refuse to admit they were wrong.

  skarz (talk) 17:48, 24 December 2023 (UTC)

• Originally I was gravitating towards Option 2 for the same reasons as S Marshall and InvadingInvader (though I do not share the latter's views about BLP): already must all content be checked for accuracy, policy compliance, etc., prior to hitting the "save edits" button, so the main thrust of this proposal is in the disclosure requirement, which is unenforceable because there is no reliable way of detecting whether content is AI-generated. But after thinking about this some more, I realized that this proposal would essentially have us treat suspected AI-generated content the same way we do suspected WP:COI content, i.e. not outright banned but "strongly discouraged" with a requirement to disclose, especially if you are being paid to edit. Maybe that isn't such a bad thing, but I'm still not fully convinced it's necessary. If an editor is being disruptive because the content they are adding is inaccurate or has fake references, and we suspect they are using LLMs without disclosure, we can already sanction them directly for adding inaccurate content with fake references; there is no need for this disclosure policy to do that. Mz7 (talk) 10:53, 24 December 2023 (UTC)
• Option 2 The focus should not be on "who" authored the content but rather on its verifiability and adherence to Wikipedia's style and content policies. In my own research, I've frequently come across books that rely heavily on Wikipedia content. Obviously I have to set those aside to avoid a 'feedback loop' of using Wikipedia to cite Wikipedia. This realization underscores a critical point though: the source of information, whether it be a human or an LLM, is secondary to the quality and reliability of the information itself. Wikipedia has always championed the principle of verifiability. Content, regardless of its origin, must be fact-checked and properly cited. This standard applies equally to human contributors and LLMs. The introduction of LLM-generated content doesn't diminish this expectation; rather, it reinforces the need for meticulous oversight and rigorous adherence to guidelines. This is no different than if someone were you to write 10 lines of basic code: You can write it yourselgf from scratch, copy it from Stack Overflow, or ask ChatGPT to write it fro you. If the outcome is the same and the code accomplishes the task - where's the discrepency? Implementing well-structured guidelines for LLM contributions only servers to strengthen Wikipedia. These guidelines should be designed not as rigid, dichotomous policies, but as flexible frameworks that ensure content quality and reliability. skarz (talk) 17:38, 24 December 2023 (UTC)

@Skarz: According to Buidhe's formulation, option 1b is ban all LLM-generated content from the wiki. I just wanted to confirm if that is what you meant since the explanation of your choice sounds different. Phlsph7 (talk) 18:06, 24 December 2023 (UTC)

Did I post my response in the wrong place? skarz (talk) 18:24, 24 December 2023 (UTC)

No, the place is correct. I only got the impression from your explanation that you were not generally opposed to LLMs. But it's possible that I misinterpreted your explanation so please excuse the confusion. Phlsph7 (talk) 18:59, 24 December 2023 (UTC)

I am not opposed to LLMs. I thought this RFC was regarding whether not LLM-generated text should have to be attributed/cited as such, not whether LLM-generated content was prohibited. skarz (talk) 19:03, 24 December 2023 (UTC)

I agree, it has become a little confusing since some editors have suggested additional options in their votes. Currently, the most popular ones are

• Option 1: new guideline/policy that all LLM output has to be disclosed
• Option 1b: new guideline/policy that all LLM-generated content is banned
• Option 2: no new guideline/policy.

Phlsph7 (talk) 19:08, 24 December 2023 (UTC)

Huh? I sure as heck hope not, because that isn't what Option 1 says. I suppose there is no rule against making RfC comments like "Support option N I think Option N is bad and we should do something different". But I think if we want to have a referendum on a different issue, we should have a separate RfC for it. jp×g🗯️ 06:26, 26 December 2023 (UTC)

I'll note that the previous draft (WP:LLM) met with ignominious failure after people attached a litany of additional pet-peeve provisions onto it, resulting in a huge trainwreck referendum-about-everything-in-the-world on which it was impossible for consensus to develop (and indeed, none did, meaning that we spent months with no guidance whatsoever). jp×g🗯️ 06:31, 26 December 2023 (UTC)

• Option 1(a), as policy – editors must disclose both their LLM use and the specific model used, as well as guarantee that they have manually checked its entire output, making them liable for accuracy, style, suitability, and plagiarism issues as per usual. Since clarity is the most important aspect here, failing 1(a) my second choice would be 1(b), disallowing LLM use altogether – because if we do not adopt disclosure, I do not think LLMs should be used transparently onsite. Remsense留 21:10, 24 December 2023 (UTC)
• option 1 policy.'..ban LLM content. similar to plagiarizing Daily Mail. Ever notice admins acting in BIG corp. or nation-state interests? 3MRB1 (talk) 04:53, 23 December 2023 (UTC)

  No, not really. In my opinion, your third sentence is a bit of a non-sequitur. –Novem Linguae (talk) 05:52, 23 December 2023 (UTC)

• Option 1 if we are going to allow them then we should have this sentence in. Emir of Wikipedia (talk) 15:15, 24 December 2023 (UTC)
• Option 1 (as policy) given the massive verifiability issues caused by LLM output, including the often completely made-up citations giving an illusion of verifiability. I am open to restricting the policy to new prose written by LLMs (rather than their use as assistance for, e.g., translation), as these are the most egregious cases and should be banned. ChaotıċEnby(t · c) 07:05, 25 December 2023 (UTC)

  The problem with the proposed policy is that there is no limit of the restriction on LLM. We all may agree if LLM is banned on new prose, but how about the use of LLM for paraphrasing? Or fix grammatical mistake? Translation? Adding proper semicolons? ✠ SunDawn ✠ (contact) 15:54, 25 December 2023 (UTC)

• Option 2. There no point in banning all LLMs or requiring disclosure since all edits either pass off as an AI edit or as a real human edit. In the former case nothing is preventing another editor from reverting said edit, and in the latter case people just won't disclose them if they are acting in bad faith. What is the point then, of a policy/guideline that only harms good faith editors, whose edits will be put under unwarranted scrunity simply because they were using a LLM, or worse, can't use a LLM at all because people have decided it can't do good at all? 0xDeadbeef→∞ (talk to me) 09:18, 25 December 2023 (UTC)

  I don't think this is true. LLM edits in and of themselves could easily have a proper and expected place onwiki. Of course people will just ignore the policy—the reason it would be a worthwhile policy would be its applicability when chronic issues crop up in addition to the specific content issues improper LLM use creates—it's not bad to have a "redundant" policy per se. If users disclose, it should not be used to just blindly revert LLM edits, that wouldn't be acceptable. Rather, it could be used to help editors who may be making incidental errors with their use, or perhaps even to mark LLM edits so over time we more readily accumulate more data about what LLM is good for on Wikipedia. imo Remsense留 19:57, 25 December 2023 (UTC)

  Without building it into the edit summary fields (similar to the minor edit tickbox), a guideline/policy requiring disclosure would be useless since new users (the ones that need more help in understanding LLM and their suitability for Wikipedia content) don't know about it. 0xDeadbeef→∞ (talk to me) 03:26, 26 December 2023 (UTC)

  Maybe it would be worthwhile writing into the blurb that appears on the edit page? Not that a lot of new editors always see that, but it's there. Remsense留 18:31, 27 December 2023 (UTC)

  LLM edits in and of themselves could easily have a proper and expected place onwiki - in case you were responding to me about the edit being perceived as either being an AI edit or as a human edit, as I realize it is a bit unclear: the point is that an edit is either blatantly AI (bad grammar, long paragraphs with a specific tone without references, etc.) which is already problematic. If a LLM edit is convincingly human already, then what's the point of such a guideline/policy? 0xDeadbeef→∞ (talk to me) 13:27, 30 December 2023 (UTC)

• Option 2; humans are just as capable of creating unsuitable content as machines, just not as quickly. I don't think it helps to say "A computer helped me write this thing which shouldn't be on Wikipedia" if it's going to be removed based on its content alone. Especially if it's undetectable either way and anyone who wants to is capable of not labelling their edits. The burden is always on the editor to check and verify what they're publishing, no matter where it comes from. HerrWaus (talk) 03:42, 26 December 2023 (UTC)

  I've realised I'd agree to the inclusion of the sentence without the part about disclosure. I would change it to "Large language model output, if used on Wikipedia, must be manually checked for accuracy (including references it generates); text added in violation of this policy may be summarily removed." This is with the understanding that anything checked for accuracy and found to be lacking would be summarily removed anyway. HerrWaus (talk) 03:50, 26 December 2023 (UTC)

• Option 2; why does LLM content need to be manually checked by a human to be added, but not manually checked by a human to be removed? Why should use of LLMs to translate have to be reported but not use of Google Translate? It is much easier to read a sentence in your non-native language and verify if it is correct than to produce a sentence in the first place. LLM's can help non-native speakers produce better prose and discouraging its use might decrease the quality of their prose or even discourage them from editing. Technologies that assist non-native speakers in contributing should be encouraged to help counteract the bias from having an editorship mostly composed of native speakers. Photos of Japan (talk) 14:39, 26 December 2023 (UTC)
• Option 2. I find both the review and notice requirements of this proposal to be entirely reasonable, but the extra bit of verbiage at end about such content being per se removeable is almost certain to invite a large number of instances of editors (especially among less experienced users) engaging in self-propelled analysis of what may or may not be machine-derived content, rather than invoking this rule only when there is an admission/delcaration of such tools being used. This would be pretty much absolutely certain to open up a massive new front for ABF, edit warring, and general disruption, so the proposal is untenable with the current language, imo. SnowRise ^{let's rap} 17:59, 26 December 2023 (UTC)

  Imagine a professional translator using an LLM (as many of them do) to generate rough drafts, and spending a few months on Wikipedia translating articles from another language into English, unaware that an LLM policy even exists. One day they casually recommend an LLM to another user, stating it is what they use to generate rough drafts to improve their workflow. Another user notes they never disclose in their edits that they use an LLM, and mass reverts them. The translator doesn't bother defending themselves but just silently leaves Wikipedia, frustrated that months of translations were erased. It's not hard imagining scenarios where this policy causes significant issues. Photos of Japan (talk) 03:21, 27 December 2023 (UTC)

  I've had the misfortune of copy editing a machine translated Wikipedia article before. I was not told it was machine translated until later. It was quite a pain. It was fluent-sounding, but every sentence needed fixing, and sometimes the original meaning was not guessable. I am not sure that using AI is a great workflow for translators. –Novem Linguae (talk) 08:34, 27 December 2023 (UTC)

  In a previous State of the industry report for freelance translators, the word on TMs and CAT tools was to take them as "a given." A high percentage of translators use at least one CAT tool, and reports on the increased productivity and efficiency that can accompany their use are solid enough to indicate that, unless the kind of translation work you do by its very nature excludes the use of a CAT tool, you should be using one.

  Over three thousand full-time professional translators from around the world responded to the surveys, which were broken into a survey for CAT tool users and one for those who do not use any CAT tool at all.

  88% of respondents use at least one CAT tool for at least some of their translation tasks.

  Of those using CAT tools, 83% use a CAT tool for most or all of their translation work.

  Computer-aided Translation, by all appearances, is widely used by professional translators to improve their efficiency. Photos of Japan (talk) 09:07, 27 December 2023 (UTC)

  By professionals, who both understand the limits of the technology and are aware they face additional scrutiny if their tool makes a canonical mistake. This is often distinct from what happens with machine translation onwiki.
  You're replying to someone talking about an instance of a very broad class of misuse of the tool, it doesn't help to point to the fact that the tool can be properly used by those with training. Remsense留 18:30, 27 December 2023 (UTC)

  I am replying to someone stating that they don't think using AI is good for improving the workflow of translators. This is a misperception, and it is useful to clear up misperceptions before policies are written based off of them. You don't notice translators who use AI and don't have any problems. You only notice those that do and have many problems. You cannot make a determination of the usefulness of AI-assisted translation based off these one-sided experiences. Many users here have expressed a desire to discourage LLM usage for translation. Me pointing out that people who do translation for a living widely use AI to assist them is an obviously relevant piece of information to introduce. Photos of Japan (talk) 18:53, 27 December 2023 (UTC)

  They have not said that in any of their messages in this RFC, as far as I can tell. No one is saying machine translation should not be used onwiki.
  Your initial proposition seems to oddly conflate using LLM to generate drafts and using machine translation to translate text. Could you be a bit more clear about your LLM concerns specifically? (edit conflict) Remsense留 19:16, 27 December 2023 (UTC)

  "someone stating that they don't think using AI is good for improving the workflow of translators" is a paraphrase of "I am not sure that using AI is a great workflow for translators".

  "draft" when talking in the context of a translation, refers to a draft of the translation. I'm sorry that was not clear enough for you. Also, I have not even typed "machine translation" in any of my comments, you are oddly conflating my comments with other users who have replied to me talking about machine translation. Photos of Japan (talk) 19:25, 27 December 2023 (UTC)

  To clarify my position on machine translations, I believe they are useful for reading foreign language sources and getting the gist of them, but should never be used to translate entire articles or assist in translating entire articles that end up on English Wikipedia. Translation should be done from scratch by a competent bilingual speaker. –Novem Linguae (talk) 23:40, 27 December 2023 (UTC)

• Option 1b, policy should explicitly be against LLM use. Many good points above, including the note that issues surrounding enforceability are equally applicable to many of our existing politics, and should not be a barrier to a policy decision being implemented. Requiring other editors to check the lengthy rubbish produced by LLMs is a much poorer option than stating from the outset that LLMs should not be used. There is great similarity here to WP:MACHINETRANSLATION, and despite the permissiveness there being heavily curtailed, there are still often issues with machine translations other editors end up having to deal with. CMD (talk) 05:47, 27 December 2023 (UTC)
• Option 1b as policy. And do all we can to discourage their use. LLMs are trained to produce what looks like well-written well-informed text, and they do it well enough to misinform readers, and to mislead writers who use them as a tool. But they have no understanding of language. They do not use it to convey information or understanding. An example follows.

Question put to Bard:

 How many legs do ten elephants have, if two of them are legless?

Bard's reply:

 Eight legs per elephant is the standard, so ten elephants would have 8 legs/elephant * 10 elephants = 80 legs.

 However, if two of the elephants are legless, then we need to subtract their leg count from the total. Since legless elephants have 0 legs, the total leg count becomes 80 legs - 0 legs from 2 legless elephants = 80 legs.

 Therefore, ten elephants with two of them being legless would have 80 legs.

Maproom (talk) 18:40, 27 December 2023 (UTC)

I doubt this is in any way respresentative of the kind of prompts that a reasonable editor would use when contributing to the encyclopedia. A better example of one might be something like this. Sohom (talk) 19:14, 27 December 2023 (UTC)

I tried again using the same prompt:

Bard's reply:

Eight-legged elephants are a fun concept, but in reality, an elephant has four legs. So, even if two elephants were missing all their legs (which is not possible for an elephant), the remaining eight elephants would still have a total of 32 legs.

Chat GPT reply:

Elephants typically have four legs each. If two of the ten elephants are legless, the remaining eight elephants would have a total of 32 legs (8 elephants x 4 legs/elephant). The two legless elephants would not contribute any legs to the total count.

Bard seems to be still daydreaming as nobody tells them that elephant has 8 legs, but ChatGPT provided an accurate answer. ✠ SunDawn ✠ (contact) 02:22, 28 December 2023 (UTC)

• Policy option 1b. All articles should be written by humans. LLMs should be discouraged and any LLM content should be treated as unreferenced or improper by policy. Or something like that, along the lines of Option 1b. Andre🚐 06:00, 28 December 2023 (UTC)
• Option 1 – policy, not guideline with the following changes: The use of Llarge language model(LLM) output is discouraged on Wikipedia. , iIf LLM output is used on Wikipedia, it must be manually checked for accuracy (including references it generates) and copyright violations (e.g., plagiarism), and its use (including which model) must be disclosed by the editor; text added in violation of this policy may be summarily removed. I think we should discourage LLM use for its many issues, particularly hallucinations, but banning it is utopian since we have no good way of reliably detecting it (and I doubt that we will). I've added the part about copyright violation to address one of the major concerns with LLMs, which is plagiarism. Option 1 only works as a policy because, as phrased, it is mandatory and there are no exceptions noted in the draft. voorts (talk/contributions) 01:53, 30 December 2023 (UTC)
• Option 1a LLMs are potentially walking copyright violation machines. We should at the very least keep track of likely-tainted material in case a legal precedent is set which rules its use to be copyright infringement as a whole. I like Voorts' phrasing on this. Acebulf ^{(talk | contribs)} 02:20, 30 December 2023 (UTC)
  • @Acebulf and Voorts: Copyright infringement is already prohibited by existing policies, which (rightly) dont distinguish between LLM and human generated copyvios so if you see anything you know or suspect is a copyright violation then it can be (and should be) dealt with under existig processes. Similarly all the other problems that LLMs can (but note don't always) produce can and should be dealt with by existing processes. This proposal would not serve to track LLM-generated content because there is no reliable way to determine what is and is-not LLM-generted and even if it was mandatory to disclose the use of LLM, most people who would use it will either not be aware of the need to disclose or will intentionally not disclose (and see also the discussion section where it is explained how every mobile phone edit may need disclosure). Additionally, there is no specific method required for disclosure so you would need to search the text of all edit summaries, (user) talk page messages and for hidden comments in the text for anything that mentioned "LLM", "large language model", any synonmys, and the names (including abbreviations and nicknames) for every LLM. Additionally you'd need to look for misspellings, typos and foreign-language versions of all the names too. Finally, even if you did manage to generate a list, and dealt with all the false positives and false negatives, there is still no guarnatee that any or all the LLM-content remains in the article due to the normal editing process. Thryduulf (talk) 11:55, 31 December 2023 (UTC)
• Option 1b Writing an article using AI is starting from the wrong end. Articles should be written starting from individual, specific reliable sources, not an AI average. We need to emphasise this to new editors as a core value of how people edit Wikipedia, otherwise we risk articles filling up with errors.
  I have tested AI on topics I know a lot about. AI platforms do not have sufficient knowledge of obscure topics to write reliable articles at this time. They mix up topics that are similar but not the same and produce text that sounds very very plausible but is totally wrong. My tests on more obscure topics tend to run at about 30-40% of statements containing errors of some kind.
  If you have the knowledge to write a good article, you can do it without an AI. If you don't, you aren't going to be able to spot where the AI is getting things wrong. If there's enough training material for an AI to write a good article, it's probably a topic we already have an article on. Either way, it should be the writer creating the article, in order that they think through what they're putting into Wikipedia and not copying and pasting. Blythwood (talk) 06:57, 31 December 2023 (UTC)
• Option 1 -- Policy. LLMs are not sufficiently reliable yet. --A. B. ^{(talk • contribs • global count)} 22:47, 31 December 2023 (UTC)
• Option 1 as policy, and this should be considered as support for the lesser included versions of option 1 if only one of those achieves consensus. I acknowlege the difficulty in detection and enforceabilty, but I do think that a relatively bright line rule is needed for dealing with problematic cases. Xymmax ^{So let it be written} _{So let it be done} 22:26, 6 January 2024 (UTC)
• Option 1 or 1a guideline, strong oppose 1b. AI currently seems incompetent at writing encyclopedic articles and its input should be carefully checked. However, we should not bar the minority of AI-generated content that is competent and helps to build an encyclopedia. As well, AI is constantly evolving, and we should not slam the door shut on emerging technology. I personally don't care whether or not it explicitly condemns it or if it's a policy. ^Queenof_Hearts 01:11, 8 January 2024 (UTC)

  (Did you mean 1a, strong oppose 1b?) Remsense留 01:42, 8 January 2024 (UTC)

  Derp. ^Queenof_Hearts 23:36, 15 January 2024 (UTC)

• Option 2 If such a Luddite attitude had been in place at the outset, then Wiki technology might have been forbidden too and so we'd be stuck with Nupedia which faltered and failed. I recently came across a new article which starts to explain how Google is experimenting with LLMs to control household robots and building in security protocols to constrain and control them. As LLM algorithms are built into such general purpose technology, we will encounter them in a varied and evolving way and so should keep an open mind about the possibilities and potential. Andrew🐉(talk) 18:31, 8 January 2024 (UTC)

  I agree that we should keep an open mind, but the technology in its present state does not achieve adequate results. If that changes, the policy can change too. Remsense留 20:40, 8 January 2024 (UTC)

  We should be blaming the results, not the technology. LLMs are quite adept at fixing grammatical mistakes, should their inability to create a good article precludes them from being used as a grammatical checker tool? ✠ SunDawn ✠ (contact) 02:12, 9 January 2024 (UTC)

  I don't think so, hence my preference for 1(a). Remsense留 02:13, 9 January 2024 (UTC)

  The RfC does not have an option 1(a). We see from such chaotic drift that human input is imperfect. Every page of Wikipedia has a disclaimer saying that the results of our work should not be trusted. So, what's being presented here is a false dichotomy. It's not perfect human work vs imperfect machine output. It is all much the same. Andrew🐉(talk) 09:46, 9 January 2024 (UTC)

  Almost the entire length of the RfC, the distinction has been made. It's not a "chaotic drift"—it's a very normal drift that happened for describable, human reasons, much unlike the output of an LLM, where it will lie to your face if you ask it why it gave you the output it did. That's the key distinction here: you can work with humans, you cannot work with LLMs. It is somewhat incredible in the literal sense to insist that the dichotomy between human and LLM output is a false one. Remsense留 09:51, 9 January 2024 (UTC)

  Someone else says below that "There doesn't appear to be an "Option 1a" anywhere". The fact that we can't even agree what the options are demonstrates what "working with humans" is like. Especially on Wikipedia where we are officially encouraged to "Ignore all rules". Andrew🐉(talk) 10:03, 16 January 2024 (UTC)

  I can safely agree with ≥75% of the people who've engaged in this RFC what 1(a) and 1(b) are, while I can safely agree with 0% of LLMs on such a point, even if it's contained in their training data. — Remsense诉 10:11, 16 January 2024 (UTC)

• Option 1 Requiring disclosure is a great idea, even just thinking from the perspective of copyright and ensuring that Wikipedia's text is reliably free. Leijurv (talk) 08:47, 11 January 2024 (UTC)
• Option 1 as a guideline for a fixed period before review, removing text on summary deletion. I liked the comments of Freedom4U and many others, but I echo Levivich and their thread with Epicgenius that note we can get at least some kind of data on LLM and other text-gen tool use (like MT) now, and we can also see what happens with it following a self-reporting guideline. It'll probably all be unusably messy as things are evolving so fast, but at least we can get a start on trying to gauge who's using these tools (or rather the change in and types of usage). We're also continuing the standard that you cite your sources for the material you publish as comprehensively as possible. (I think this is a net social good that is important to continue as a new tool like AI sees ubiquitous use quickly -- I can't find research specifically on this (seems like low-hanging fruit) but it feels to me like WP has raised the population's general expectations for verifiability of information at minimum.) I also want to stress that imo AI tools should be embraced here, as they largely have been already where we don't think of it as readily. I also embrace strong sources and expert editors -- I still have to properly cite them . SamuelRiv (talk) 03:56, 15 January 2024 (UTC)

Discussion

Question Does the disclosures include cases where LLMs were used for paraphrasing help ? Sohom (talk) 22:31, 13 December 2023 (UTC)

I believe that's the intention. An error might be introduced by the paraphrasing work, after all.

Unfortunately, there doesn't seem to be any reliable way to detect (or prove) violations. If this passes, you could go revert any addition you want, and say "Well, I thought it was a violation of this rule, and the rule says it can be summarily reverted". WhatamIdoing (talk) 00:59, 14 December 2023 (UTC)

Yes, that is definitely a weird loophole. I've personally used LLM outputs as inspiration for paraphrasing/rewriting attempts, and having to declare all of them/have them be reverted for no apparent reason is not really something, I'm willing to support. Sohom (talk) 01:33, 14 December 2023 (UTC)

Editors absolutely have to be responsible for what they post, but even for the most benign uses, I really wonder how many people are actually able to say "which model" they used. We have editors who aren't really sure what their web browser is. WhatamIdoing (talk) 01:47, 14 December 2023 (UTC)

Even something like "used Bing Chat" would be useful to identify LLM content, although I'd certainly prefer more detail for the prompt used or specific model (when used in a Direct Chat that lists it). TROPtastic (talk) 02:38, 14 December 2023 (UTC)

My thinking on this is, basically, that the playing field as it stands is very uneven. Prompting is a delicate art that can take a while to set up (and tokens often cost money), but nonetheless, a language model can generate a paragraph in a couple seconds. When I do GA reviews or proofread Signpost articles, I take a heck of a lot longer than a couple seconds to go over a paragraph (maybe a couple minutes, maybe half an hour if I have to look something up in a field I'm not familiar with). Normally, the system we have on Wikipedia is somewhat balanced in this respect -- it takes a while to review that a paragraph is legit, and it also takes a while to write a paragraph. While it's not perfectly balanced, it's at least within an order of magnitude. With language models, however, it's possible to create a quite large volume of text with virtually zero input, all of which (under our current policy) is ostensibly required to be treated with the same amount of delicate surgical care as paragraphs written through the hard work of manual effort.

Another thing that's important is the ability to separate people who put a lot of work into the process (i.e. multi-shot prompting, multiple runs, lorebook-style preparation) from people who are literally just typing "Write a Wikipedia article about XYZ" into the box and copypastaing whatever pops out into the edit window. The first group of people, which includes me, is responsible for stuff like the Signpost templates functioning properly and not having busted CSS (thanks GPT-3.5). The second group of people is a nuisance at best and a force of destruction at worst. If someone is writing paragraphs of text and can't be arsed to figure out what website they got them from, why should we spend minutes or hours going through each sentence of that text individually on the assumption that it's legit? jp×g🗯️ 09:53, 14 December 2023 (UTC)

I share the concern, but the second group will probably not disclose it anyway, so the disclosure requirement is unlikely to help with cleanup. We'll have to continue relying on practices such as removing unsourced content, verifying sources, etc. If the problems are severe enough, such as using fake sources, blocks can be served (there are precedents already). MarioGom (talk) 23:04, 14 December 2023 (UTC)

I think it might be worth clarifying in the policy itself, in the event of a disclosure policy, that LLM would not itself be a reason for reversion, akin to all analogous policy. The editor is explicitly guaranteeing that they have vetted the accuracy and suitability of the LLM-generated text, so if a reversion is necessary it is just due to the normal reasons one would revert an edit. Remsense留 21:06, 24 December 2023 (UTC)

User:JPxG, is this based on this suggestion by User:Tamzin in the prior RFC? WhatamIdoing (talk) 01:01, 14 December 2023 (UTC)

Great minds, etc etc -- this page is from Sep 1, that comment is from Sep 6, but there has certainly been some intellectual confluence since then. jp×g🗯️ 01:16, 14 December 2023 (UTC)

Perhaps out of boredom, I asked a large language model (GPT-3.5) to write an essay in support of this policy and an essay in opposition to this policy. I've included them below in a collapsed column if anyone is interested:

In support

Large language models like GPT-3 possess remarkable capabilities to generate text across diverse subjects, offering unprecedented assistance to editors contributing to platforms like Wikipedia. However, their usage on such platforms demands responsible oversight to maintain the integrity and accuracy of information. Therefore, implementing a policy mandating the manual verification of language model output, including the references it generates, is crucial. This measure ensures that the information added to Wikipedia is reliable and aligns with the platform's standards. Firstly, while large language models can produce coherent and seemingly accurate content, they lack the discernment and fact-checking abilities inherent in human editors. Verification of their output becomes imperative to sift through the vast array of generated text and validate the accuracy of the information and references provided. This manual review serves as a safeguard against the inadvertent propagation of misinformation or biased content, maintaining Wikipedia's reputation as a reliable source of information. Secondly, disclosure regarding the use of language models in contributing to Wikipedia is essential for transparency. Editors should openly acknowledge the utilization of such technology and specify the model employed. This disclosure not only fosters transparency but also allows for better monitoring and evaluation of the content generated. It enables the community to understand the potential sources and methodologies used, promoting accountability and trust among editors and readers alike. Consequently, any content added in contravention of these guidelines should be subject to removal from Wikipedia. Upholding strict policies ensures the platform's credibility, as information obtained from unchecked language model output poses the risk of inaccuracy or bias. By promptly removing such content, Wikipedia maintains its commitment to providing accurate and reliable information, upholding its status as a reputable knowledge-sharing platform. In conclusion, while large language models offer incredible potential to aid in content creation, their utilization on platforms like Wikipedia demands stringent oversight. Mandating manual verification of generated text, disclosing the use of language models, and promptly removing content added in violation of these policies are essential steps to preserve the integrity and accuracy of information on Wikipedia. These measures ensure that the platform continues to be a reliable and trustworthy source of information for millions of users worldwide.

In opposition

While the integration of large language models like GPT-3 in content creation offers a revolutionary approach to generating information, imposing stringent manual verification and disclosure requirements on their usage for Wikipedia may pose several counterproductive challenges. First, the sheer volume of data these models produce is immense and manually verifying each output, including references, becomes an impractical task. The time and resources required for such meticulous verification could significantly impede the pace of content creation and hinder the platform's ability to keep up with the constantly evolving information landscape. Furthermore, mandating disclosure of which model an editor uses and the generated text's origin might inadvertently lead to bias against certain models or methodologies. Editors might face unwarranted skepticism or prejudice based on their choice of model, potentially discouraging them from using innovative tools that could otherwise enhance the quality and diversity of content. This limitation might stifle the exploration of new technology within Wikipedia, impeding the platform's ability to adapt and evolve in the rapidly advancing field of information generation and dissemination. The strict removal of content added without adhering to these stringent policies could also hinder the collaborative nature of Wikipedia. It may discourage contributions from well-intentioned editors who, while using language models, might inadvertently miss the manual verification or disclosure steps. This approach risks alienating contributors and potentially diminishing the wealth of knowledge that diverse perspectives and contributions bring to the platform. In conclusion, while ensuring accuracy and transparency are paramount in maintaining Wikipedia's credibility, imposing rigid manual verification and disclosure requirements for language model usage might create more challenges than solutions. Striking a balance between leveraging these technological advancements and maintaining editorial standards is crucial to foster an environment that encourages innovation, inclusivity, and the continuous growth of accurate and reliable information on Wikipedia.

— Red-tailed hawk _(nest) 03:28, 14 December 2023 (UTC)

It's amusing that the Oppose essay is not because the proposal could removal human content too hastily, but because it attempts to control LLM use at all. "The time and resources required for such meticulous [manual verification of output including references] could significantly impede the pace of content creation" indeed... TROPtastic (talk) 07:27, 14 December 2023 (UTC)

I especially loved its final sentence: I was waiting for it to say "synergy" and "paradigm".

I was curious at how bad the "Oppose" essay prompt was, considering the "Support" essay was decent enough -- at least in-line with the policies and how people generally interpret them. So I asked GPT-4 via MS Copilot to write a short essay in opposition to the policy change as written:

Opposition to Proposed Policy

Wikipedia is a free online encyclopedia that anyone can edit. It aims to provide reliable and verifiable information on a wide range of topics. However, some editors have suggested that Wikipedia should adopt a new policy that would restrict the use of large language models (LLMs) on the site. LLMs are artificial intelligence systems that can generate natural language text based on a given input. They can be used to create or improve Wikipedia articles, especially on topics that are underrepresented or lack sources. However, the proposed policy would require that any text generated by LLMs must be manually checked for accuracy and disclosed by the editor, and that any text added in violation of this policy may be summarily removed. This policy is unnecessary, unfair, and harmful to the quality and diversity of Wikipedia. First, the policy is unnecessary because Wikipedia already has mechanisms to ensure the quality and verifiability of its content. Wikipedia has a set of core policies and guidelines that all editors must follow, such as citing reliable sources, maintaining a neutral point of view, and avoiding original research. Wikipedia also has a community of editors who can review, edit, or revert any changes made by other editors, including those made by LLMs. Wikipedia also has various tools and bots that can detect and flag vandalism, spam, or other problematic edits. Therefore, there is no need to single out LLMs as a special case that requires additional scrutiny and disclosure. Second, the policy is unfair because it would create an unequal and discriminatory environment for editors who use LLMs. LLMs are not inherently good or bad, but rather depend on how they are used and for what purpose. LLMs can be used to generate text that is accurate, informative, and helpful, or text that is inaccurate, misleading, and harmful. The same can be said for any other source of information, such as books, websites, or human experts. Therefore, it is unfair to impose a higher burden of proof and transparency on editors who use LLMs than on editors who use other sources. Moreover, the policy would discourage editors from using LLMs, even if they could improve the quality and diversity of Wikipedia articles. This would create a disadvantage for editors who have limited access to other sources, such as those from developing countries, minority groups, or marginalized communities. Third, the policy is harmful because it would reduce the quality and diversity of Wikipedia. LLMs have the potential to enhance Wikipedia's mission of providing free knowledge to everyone. LLMs can help create or improve articles on topics that are underrepresented or lack sources, such as those related to women, people of color, indigenous cultures, or emerging fields. LLMs can also help generate text that is more readable, engaging, and accessible to a wider audience, especially for non-native speakers or people with disabilities. By restricting the use of LLMs, the policy would limit the scope and depth of Wikipedia's coverage, and undermine its goal of being a comprehensive and inclusive encyclopedia. In conclusion, the proposed policy to regulate the use of LLMs on Wikipedia is unnecessary, unfair, and harmful. It would create an unequal and discriminatory environment for editors who use LLMs, and reduce the quality and diversity of Wikipedia's content. Therefore, the policy should not be adopted as a policy or guideline, and instead, Wikipedia should encourage and support the responsible and ethical use of LLMs to improve its encyclopedia.

Honestly, in many respects it makes pretty much the case I make. SamuelRiv (talk) 02:54, 15 January 2024 (UTC)

• Question: Would an edit summary suffice for disclosure of LLM use? Should an example edit summary be added to the policy? Maybe a checkbox akin to the "minor edit" checkbox? Dialmayo (talk) (Contribs) she/her 16:15, 14 December 2023 (UTC)

  I'd be very hesitant for any checkbox or interface changes because that de facto encourages people to use GPT (which is definitely not what we want here, and a WP:BEANS violation). Otherwise, I'm guessing we would treat LLM use declarations in the same way we do COI. Fermiboson (talk) 16:18, 14 December 2023 (UTC)

  So like an edit request on the article with a template like {{edit COI}}? In any case, this probably warrants a separate discussion if/when the policy passes. Dialmayo (talk) (Contribs) she/her 18:47, 14 December 2023 (UTC)

  See [2]. Dialmayo (talk) (Contribs) she/her 19:02, 14 December 2023 (UTC)

  I would not suggest a checkbox. That would put this checkbox on thousands of wikis, introduce complexity to the MediaWiki core code, and open the floodgates for adding a bunch of other checkboxes such as "copied from within Wikipedia", etc. –Novem Linguae (talk) 20:45, 14 December 2023 (UTC)

  A "copied from within Wikipedia" checkbox might not be a bad idea... ~ F4U (talk • they/it) 04:04, 16 December 2023 (UTC)

  As I note in a previous comment, a general "oversight" flag checkbox, with more specific tagging within the edit such as translation or verification requests, might be interesting to test out on a trial basis. Sometimes it's worth noting that anotber editor may want to give a second look over something you worked on. SamuelRiv (talk) 03:19, 15 January 2024 (UTC)

• I think we need a complete ban on using this output in mainspace at present; we entirely ban the use of the Daily Mail as a source for occasionally making things up, why should the same not apply to LLM output? And to address Rhododendrites's point above, editors should not be using Google Translate to put material in article space either. However I agree that if one could summarily remove text suspected to be LLM generated, it would allow removal of any disliked text, in the same way that asserting an editor must be paid is a great way of making any new editor's contributions go away. Espresso Addict (talk) 01:05, 15 December 2023 (UTC)
  • we entirely ban the use of the Daily Mail as a source for occasionally making things up, why should the same not apply to LLM output - Because the scope of that analogy is wildly disproportionate. You're comparing a vast, rapidly evolving, multifaceted technology that does a wide range of things and... a single website that does nothing other than publish stories we can evaluate on the basis of RS. You're proposing banning a tool, not a source. Dozens of companies/organizations are making them with different strengths, weaknesses, purposes, etc., and any one of them can do a wide range of things related to language. Here we're flattening all of it and just saying "no" with no nuance whatsoever? — Rhododendrites ^talk \\ 01:40, 15 December 2023 (UTC)

  Because Daily Mail has human oversite and they dont follow journal guidelines Cray04 (talk) 12:18, 30 December 2023 (UTC)

• Question I am probably missing something, but what are options 1a and 1b? Are they "promote to policy" and "promote to guideline", respectively? HouseBlaster^talk 17:51, 16 December 2023 (UTC)

  @HouseBlaster:

  • Option 1a* appears to merely be the way that HistoryTheorist has labeled her !vote (i.e. Option 1 with the caveat that I believe that suspect text should only be removed if there is overwhelming evidence of gross inaccuracy or violation of WP policies written by an LLM).
  • Option 1b appears to be phrased alternatively as We should not use LLMs at all for content pages (Awesome Aasim) or ban all LLM-generated content from the wiki (Buidhe).

  There doesn't appear to be an "Option 1a" anywhere. In any case, this sort of labeling appears to not be related to whether or not this should be treated as a policy vs as a guideline. — Red-tailed hawk _(nest) 00:35, 18 December 2023 (UTC)

  When I said option 1a -- I meant it as a policy. But my elaboration might contradict my !vote. ❤HistoryTheorist❤ 02:45, 19 December 2023 (UTC)

• Question: Has someone informed Wikipedia:WikiProject Guild of Copy Editors and Wikipedia:Typo Team about this RFC? The fact that they may soon have to disclose if they use spellcheckers like Grammarly would probably be quite interesting to them. The same applies to the editors at Wikipedia:Basic copyediting and Wikipedia:Spellchecking since these pages currently recommend the usage of spellcheckers. Phlsph7 (talk) 14:15, 19 December 2023 (UTC)

  Feel free to inform any talk pages you'd like using {{subst:Please see|Wikipedia talk:Large language model policy#RFC}}. Is Grammarly an LLM? I don't know much about that software. If it is, probably wouldn't hurt for those folks to change their edit summaries to something like "typo fixing using Grammarly". –Novem Linguae (talk) 19:43, 19 December 2023 (UTC)

  Done, thanks for the tip. I'm no expert but according to my interpretation of [3], that's the technology underlying the error detection of Grammarly. LLMs are being integrated more and more into common software, including word processors like Microsoft Word. If the suggestion in the current wording passes, properly informing editors exactly what needs to be disclosed might be a challenge. Phlsph7 (talk) 08:44, 20 December 2023 (UTC)

  I use the default keyboard on my Android phone. It has recently started suggesting grammar changes by default, without my changing any settings. I have no idea what technology is used for this, whether there is any AI involved or how I would find out. If there is an AI involved and I accept its suggestions does this mean I have to disclose this? The wording of the proposal indicates I should. What about if I consider the suggestions but choose not to implement them? This implies that every edit made using a mobile phone should be suspected of having AI involvement, and every page edited that way would need to be tagged, and every user who does not explicitly disclose (whether they know they are using AI or not) would be subject to potential sanction. This makes it even more clear that this proposal is not only pointless but also actively harmful. Thryduulf (talk) 03:04, 22 December 2023 (UTC)

  According to [4], autocompletion features rely on "language models" for their predictions. The page also discusses "large language models" but it does not clarify whether the language models commonly used for autocompletion are large enough to qualify as large language models. As I understand it, large language models can be used for autocompletion. The question of whether a particular autocompletion feature relies on a large language model would require some research and mobile users are usually not aware of the underlying autocompletion technology. Given these difficulties, it might be necessary for all mobile users who have autocompletion enabled (which is usually enabled by default) to disclose this for every edit. I agree with you: this does much more harm than good. Phlsph7 (talk) 08:47, 22 December 2023 (UTC)

  Similarly, search engines use AI. I don't think we need editors to disclose that they used a search engine for research. Levivich (talk) 15:46, 22 December 2023 (UTC)

  I think it goes without saying that we don't need to ban or increase disclosure requirements for search engines and typo correction tools. If there is concern about this proposal being used to crack down on such things, perhaps the proposal's wording should be altered slightly to carve those out. The meat and potatoes of this proposal (to me) seems to be the use case of asking ChatGPT to write a sentence-length or greater block of text, then using that generated content in articles. –Novem Linguae (talk) 21:26, 22 December 2023 (UTC)

  Before you do that, perhaps you could identify some problem with LLM-generated text that cannot be dealt with using existing processes - despite multiple people asking multiple times in every discussion to date, nobody has yet done this. Thryduulf (talk) 21:35, 22 December 2023 (UTC)

  I think the advantage of an LLM PAG is clarity. If we tell new users "we don't allow LLMs" or "LLM usage must be disclosed", it's very clear. If we tell new users "we don't allow hoaxes", "citations must not be fake", "citations must support the text they're associated with", "statements must be factually correct", etc., that's great and all, but they may not even know that LLMs create these kinds of issues. –Novem Linguae (talk) 21:50, 22 December 2023 (UTC)

  Wikipedia doesn't need a policy or guideline, though, to explain issues with text generators. It can have an explanatory essay, like the one that currently exists at Wikipedia:Large language models. isaacl (talk) 21:54, 22 December 2023 (UTC)

  This point has come up several times in the discussions and I don't think any concrete examples have been suggested where we need a new guideline/policy to remove problematic LLM-generated text that is not in violation of other guidelines/policies. I would suggest having a guideline/policy nonetheless but more for the sake of convenience. In particular, I'm thinking about cases where reviewers have to spend a lot of time removing problematic content that was created in a matter of minutes, for example, because they have to engage in AFD discussions for copy-pasted LLM original content. But this can only work in cases where the content is obviously created by LLMs, for example, because the editor admits it, because they copied the prompt together with text, or because the text contains stock phrases like "As a large language model trained by OpenAI,...". Phlsph7 (talk) 09:08, 23 December 2023 (UTC)

  Yes, I do think that the community's viewpoints would be better reflected with guidance that is abstracted away from specific implementations. Spell/grammar checkers and word completion tools aren't, as far as I can tell, causing any concerns, regardless of the tech used to implement them. Text generation is where the concerns lie, whether that with a large language model used by one tool, or some other tech in another. isaacl (talk) 21:58, 22 December 2023 (UTC)

  Agreed, text generation is the main point that needs to be addressed. I suggested the alternative formulation Original content generated by large language models or similar AI-based technologies... but I would also be fine with a formulation using the term "text generation". Phlsph7 (talk) 08:55, 23 December 2023 (UTC)

  @Novem Linguae: I agree that the generation of new text is the main thing that needs to be regulated. However, I'm not sure that this part of the formulation is a trivial matter that can be handled as a mere afterthought. There are many possible usages that need to be considered, like brainstorming, reformating, error correction, paraphrasing, summarizing, and generating original content. There seems to be wide agreement that the last point is problematic. But there could be disagreement about where to draw the line between the other usages. Phlsph7 (talk) 08:51, 23 December 2023 (UTC)

• I would urge people to avoid introducing yet more options into the consultative poll above as the risk is that doing so will generate a no consensus outcome leading to the status quo being maintained. Failing that, please explicitly name all alternatives you would also support. Stifle (talk) 09:30, 21 December 2023 (UTC)

  Thank you for the suggestion – I do think a "1(a), if not then 1(b)" response is very different from a "1(a), if not then 2" response. Remsense留 20:03, 25 December 2023 (UTC)

GPT3.5 lacks significantly in reasoning and logic, I'm sure you'll get drastically different results if you were to use GPT4. skarz (talk) 17:31, 24 December 2023 (UTC)

• Question: What is the purpose for asking people to disclose which model they are using? Say people use GPT3, GPT4, or some other model. How will that information be used? Photos of Japan (talk) 13:29, 11 January 2024 (UTC)

  It would likely be rather useful to be able to compare and contrast the qualities of the output of different models, potentially with an eye towards future policy. Remsense留 19:25, 11 January 2024 (UTC)

  Can you give an example of how you believe such information could be used for policy? LLM's are rapidly changing, while policies tend to get fossilized. I can imagine a policy discussing a particular model becoming outdated within a year once a newer model becomes free, and the policy being difficult to update due to consensus being hard to establish. Photos of Japan (talk) 11:08, 12 January 2024 (UTC)

  Does "if there are identifiable problematic patterns in the output of a given model, that model can be specifically proscribed" suffice?
  "Things are moving too fast to have rules, man" is not the compelling argument many people think it is. If the technology is evolving so fast that we simply cannot meaningfully keep track of it, then we should absolutely have a blanket prohibition on its use until things slow down and we can make heads or tails of what the technology actually does. Remsense留 12:22, 12 January 2024 (UTC)

  So you are asking people to report their LLMs in case specific problematic LLMs can be identified and banned in the future? Current LLMs will almost certainly never get a consensus to be banned, and future LLMs are expected to generally just get better. This policy as it is written is asking people to do the potentially non-trivial task of tracking down what model LLM they are using, and it isn't clear to me that there is any practical future use for this.

  I'm also not arguing "Things are moving too fast to have rules, man". I'm arguing that rules should be written in a way that they don't become quickly outdated and a burden on the community to update. This can be easily avoided by not trying to cite specific models in policy. Photos of Japan (talk) 12:21, 14 January 2024 (UTC)

  So you are asking people to report their LLMs in case specific problematic LLMs can be identified and banned in the future?

  
  Yes, in part. If editors are here to build an encyclopedia, they'll be happy to. — Remsense诉 23:51, 15 January 2024 (UTC)

  I'm here to build an encyclopedia. Give me a practical use to this information you want to require volunteers to look up and report. Specific LLM models being banned doesn't seem probable. We don't even require edit summaries in general, and those have clear explanations and arguments for why they are useful to the project. Photos of Japan (talk) 10:42, 16 January 2024 (UTC)

  Are you saying there can't be identifiable lexical patterns beholden to specific LLMs that could be useful to be able to identify and keep track of? That's the point this hinges on. Arguing that we "probably won't" target specific models in policy is flimsy. We will if one crops up that spits out a certain cultivar of garbage all the time. We have a list of perennial sources, we can have a list of perennial models. If LLMs are evolving this much, we can evolve with them.
  With normal edits there's an implicit assertion that the editor's fingers are typing or otherwise handling the words. It's straightforwardly about basic attribution. We do require edit summaries sometimes: specifically, with any other copying or sourcing not from the editor's own brain that isn't being disclosed in the wikitext itself. — Remsense诉 11:12, 16 January 2024 (UTC)

  Perennial sources is not a policy or guideline and illustrates how we don't try and ban specific sources in policy. LLM models are unlikely to ever be perennially discussed because they are constantly changing. And given that there are no examples of LLMs that have a chance to be banned, or any articulated reasons for why one could expect such a thing to arise, it doesn't seem very likely that one will. I've also never seen anyone categorically argue against any policy concerning LLMs, but people do not want policy that ignores there changing nature and that will quickly become dated and inaccurate.

  Editors copy and paste text from sources (sometimes dubious) just like they do with LLMs. There's no assertion going on with one that isn't going on with the other. People can ask a friend to review their Wikipedia article before they submit just like they can put an article into an LLM and ask it for suggestions/feedback/possible grammar errors, etc. You do not need to attribute that your friend looked over your edit and recommended some changes to it, and it's not even clear if anything an LLM produces even can be attributed to it. Photos of Japan (talk) 11:38, 16 January 2024 (UTC)

  I don't want to argue in circles, so I'll break it off with
  

  Editors copy and paste text from sources (sometimes dubious) just like they do with LLMs

  
  They shouldn't. — Remsense诉 11:41, 16 January 2024 (UTC)

  But they fact that they do contradicts your statement that:

  With normal edits there's an implicit assertion that the editor's fingers are typing or otherwise handling the words.

  I do not believe we are arguing in circles. Of course you do not need to respond. If anyone has any clear explanation for how this information can practically be used I would enjoy hearing it. Photos of Japan (talk) 12:15, 16 January 2024 (UTC)

  Note that users aren't always aware that a tool has an underlying model, and the software provider doesn't always reveal the nature of the model. isaacl (talk) 16:03, 12 January 2024 (UTC)

  Of course—I would say the most critical aspect is direct traceability to something. Additionally, users don't always know that they need to cite claims, or that close paraphrasing is copyvio. This is still a valuable policy even if a) some good-faith users provide all the information we'd like, b) some good-faith users need to be repeatedly told, and c) bad-faith users will decline to do so.

  I hope this isn't perceived as me adding a 1(c)—God forbid, to be clear I am not—or otherwise moving the goalposts. My point has consistently been that "something is much better than nothing". Remsense留 16:11, 12 January 2024 (UTC)

  For my part, it's maintaining the expectation of, as much as possible, providing attribution to any creative content that is not your own. From time to time I see pushback of the likes of 'WP is not an academic journal', but I really think (and I do hope someone does a study on this because I can't find any) that WP itself has raised the everyday individual's expectations of attribution and verifiability. We already keep such standards quite high among editors even when they are in practice optional or unenforceable. (One example I see is the overly common posts at WP:RSN that aren't disputes, but just editors looking to get proper sources on esoteric subject matter.) Giving attribution for long strings of machine-generated prose shouldn't be a chore for most editors, as long as it is said aloud to do so. I don't think it's some AI-phobic reactionary thing either -- I was shocked to learn now that editors had already been not attributing unmodified Googled Translate strings. SamuelRiv (talk) 02:45, 16 January 2024 (UTC)

  Indeed. "We're not an academic journal" (et al.) is just about the most frustrating argument one could possibly make. No one ever said we were—but everyone always says that we care about attribution and verifiability. Measures like these are what it means to actually care about those things.
  If we find LLMs—either as a whole, or certain subsets thereof—to be fundamentally unattributable, then they shouldn't be allowed. I haven't read an argument as to why it's acceptable for prose to be unattributed in this specific case and not others. — Remsense诉 03:10, 16 January 2024 (UTC)

  Until the T-Halborgylon hijacks your brain so that you are unwittingly typing AI output in good zombified faith, I don't see where there'd be a point at which strings of prose that's completely not your own is unattributable. In some weird legal scenarios (like say, there was a superinjunction on the AI you used) you can still at least say "this isn't my prose". Hypothetical for AI, but the latter kind of thing certainly has been a real issue with superinjunctions. SamuelRiv (talk) 04:25, 16 January 2024 (UTC)

  SamuelRiv, sorry for the lack of clarity—I do believe that LLM output is attributable, but there seems to be an argument as to it not being so meaningfully—which in turn seems to justify why an attempted attribution shouldn't be required. — Remsense诉 04:28, 16 January 2024 (UTC)

  I can't resist taking a moment to act superfluously pretentious and note my position is that giving any attribution to prose not entirely one's own is (additionally) a performative utterance -- it is meaningful in the act of giving attribution itself -- for all the reasons I went into above. SamuelRiv (talk) 04:43, 16 January 2024 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Discussion at Wikipedia:Templates for discussion/Log/2023 December 13 § Template:AI-generated notification

 You are invited to join the discussion at Wikipedia:Templates for discussion/Log/2023 December 13 § Template:AI-generated notification. –Novem Linguae (talk) 08:26, 14 December 2023 (UTC)

Discussion at Wikipedia:Templates for discussion/Log/2023 December 13 § Template:OpenAI

 You are invited to join the discussion at Wikipedia:Templates for discussion/Log/2023 December 13 § Template:OpenAI. –Novem Linguae (talk) 08:26, 14 December 2023 (UTC)

Notes

Future directions

I think it may be appropriate to note here my intentions for after the RfC, assuming it is successful.

When writing the proposal, I did my best to prevent it from being a "pro-LLM" or "anti-LLM" policy as written. My hope is that, rather than a meandering general referendum on the whole field of artificial intelligence, we could establish some simple and non-intrusive rule to cut down on the bottom 10% of slop without presenting too much of an obstacle to people who are interested in using the tools productively. And we are getting a rather consistent flow of slop (see WP:WikiProject AI Cleanup), from people who are either using these models improperly, using them for tasks to which they're not suited, or being insufficiently careful in verifying their output. This puts a rather large (and unnecessary) strain on new page patrollers, AfC reviewers, and editors in general.

For what it's worth, I am myself a great fan of transformer models, and have followed them with great interest for several years (I created the articles for GPT-2 and DALL-E, my first interaction with them was a GPT-2-124M in summer 2019, and I had access to the GPT-3 API in 2020). Last August I used the GPT-3 API to assist in writing several Signpost columns; I guess you will have to take my word for it that I didn't write this as a stalking-horse for a project-wide LLM ban.

Some people think that these things are just plain crap, and there is a lot of very lively debate on what utility they really have, and whether it is worth the effort, et cetera. Well, I think it is, but the consensus of the editing community isn't mine to decide, and if everyone thinks that they are junk, then I guess we will have to live with that.

I will note that the number of people who want to ban LLMs entirely increases every time a gigantic bucket of GPT slop is poured into the NPP queue, so if there's some very low-effort solution we can implement to slow down the flow, I think it is worth it even if you are a LLM maximalist who resents any sort of restriction.

Anyway, it is hard to predict the trajectory of a technology like this. They may get better, they may level off, or they may improve a lot at some things and very little at other things in a disjunct way that makes no sense. So maybe we are right on the precipice of a tsunami of crap, or maybe it already passed over, or maybe we're on the precipice of a tsunami of happiness. What I do think is important is that we have policies that address existing issues without prematurely committing to thigns in the future being good or bad. If it turns out that this cuts down on 90% of the slop and we never have an ANI thread about GPT again, then maybe there does not need to be any further discourse on the issue. If it turns out that this short sentence isn't enough, then maybe we can write more of them. jp×g🗯️ 09:37, 15 December 2023 (UTC)

Then:

• Old problem: We had a bunch of badly written articles posted.
• Old action: We wrote a bunch of rules against undisclosed paid editing.
• Old result: A few folks changed their behavior, and the rest kept doing the same thing anyway, because we had no good way to identify them.

Now:

• New problem: We have a bunch of badly written articles being posted.
• New action: We write some rules against a set of tools that might be used to make them.
• New result: A few folks changed their behavior, and the rest kept doing the same thing anyway, because we had no good way to identify them?

WhatamIdoing (talk) 04:04, 17 December 2023 (UTC)

Even if there is no good way to identify them, that does not mean it is not a bad idea to institute as policy. Is there an easy way to, for example, identify bot-like or semi-automated editing? Unless if there are tags to identify a script that made that tool, a semi automated edit could have any edit summary or no summary and no one would really know that it was semi automated. The whole point is not banning LLMs from mainspace poses a significant risk of disruption, and encouraging it would just be encouraging more disruption. And DE is one thing that, regardless of the means or intent, results in a block if it is prolonged. Awesome Aasim 22:13, 17 December 2023 (UTC)

The thing is that everything about LLM use that disrupts Wikipedia is already prohibited by existing policies. Nobody in any discussion so far has provided any evidence of anything produced by an LLM that is both permitted by current policy and harmful to Wikipedia. Thryduulf (talk) 10:27, 18 December 2023 (UTC)

Because the issue the policy is trying to address is more about larger editing patterns than individual diffs. It's not illogical if the scope of policies overlap—in fact, it's arguably a feature, since it reinforces the points that the community find most important. Remsense留 14:11, 31 December 2023 (UTC)

While there is inevitably some overlap in policies, I disagree that it's a feature per se. Generally speaking, it easier for editors to keep track of fewer policies than more, thus having a few number of central policies with supporting guidance that expands on details provides an organizing structure that simplifies remembering and following guidance. Avoiding redundancy supports this principle and helps prevent guidance from getting out of sync, and thus being contradictory. It also can forestall complaints about there being too much guidance, as the basic shape of the guidance can be understood from the central policies, and the details can be learned gradually, without having to jump between overlapping guidance. isaacl (talk) 17:04, 31 December 2023 (UTC)

I don't think that The whole point is not banning LLMs from mainspace poses a significant risk. I think there's some good old human emotions at play here, but the problem is that we already know the ban will be ineffective. Most people won't know the rule, you won't be able to catch them (and we will wrongly accuse innocent people), and most of the few people who are using LLMs and actually know the rule won't follow it, either, because a good proportion of them don't know that you decided that their grammar checker is an LLM, and the rest don't think it's really any of your business.

This is King Canute and the tide all over again: We declare that people who are secretly using LLMs must stop doing it secretly, so that we know what they're doing (and can revert them more often). You're standing on the beach and saying "You, there! Tide! Stop coming in, by orders of the king!" We can't achieve any of the goals merely by issuing orders.

And your plan for "And what if they don't follow your edict?" is what exactly? To harrumph about how they are violating the policies? To not even know that they didn't follow your orders? WhatamIdoing (talk) 07:06, 11 January 2024 (UTC)

A good summary of our WP:COI guidelines, but it doesn't seem a reason to scrap them. CMD (talk) 07:28, 11 January 2024 (UTC)

I am also concerned that it will add an unnecessary burden on those of us who will follow the policy, for no apparent reason. MarioGom (talk) 12:04, 11 January 2024 (UTC)

Request for close

I'm going to make a request, because the bot just removed the RFC template since it's been a month (I obviously am not going to close it myself). jp×g🗯️ 10:18, 13 January 2024 (UTC)