Wikipedia:WikiProject Articles for creation/RfC for AfC reviewer permission criteria

RfC for reviewer permission criteria

At Wikipedia:WikiProject Articles for creation/RfC Reviewer permission a consensus was reached to introduce a permission system for reviewers. The topics on what the permission threshold should be, and how it would be implemented, were deliberately left open.

1. This RfC discusses suggestions for the threshold of experience for users to demonstrate that they are adequately versed in the policies and guidelines involved for an article that can exist uncontentiously in mainspace.
2. This RfC does not discuss how this threshold will be granted and/or implemented. That will be the topic of a further discussion, when the threshold itself has been established.
3. This RfC is not a vote. Participants are invited to discuss what would be a reasonable threshold. The closer will assess the outcome based on the discussion.
4. Consensus has been reached for this permission, this RfC is not for rediscussing whether a permission is needed.

Background examples

• Reviewer (Pending Changes Reviewer): Quick check to ensure edits don't contain vandalism, violations of the policy on living people, copyright violations, or other obviously inappropriate content. The threshold is deliberately low but Reviewers are not expected to be subject experts and their review is not a guarantee in any way of an error-free article. They are expected to have a reasonable editing history, distinguish what is and what is not vandalism, and be familiar with basic content policies.

Reviewer permission are specified as follows:

1. You have an account, and routinely edit.
2. You have a reasonable editing history – as a guide, enough edits that a track record can be established.
3. You have read our policy on vandalism and understand what is and what is not vandalism.
4. You are familiar with the basic content policies: Biographies of living persons, Neutral point of view, No original research, Verifiability and What Wikipedia is not.
5. You are familiar with the basic legal policy: Wikipedia:Copyrights.
6. You have read the guideline on reviewing.

Permission is granted by an admin.

• Rollbacker: While there is no fixed requirement, a request is unlikely to be successful without a contribution history that demonstrates an ability to distinguish well intentioned edits with minor issues from unconstructive vandalism. Users with 200 edits (generally discounting those to their own user space) can apply for training to the WP:CVUA. Admins rarely grant the tool for less than a clear run of at least 100 reverts of clearly identifiable vandalism without errors. Significant experience is needed to identify the kind of edits that may not appear to vandalism at first sight e.g. inappropriate edits missed by the bots and abuse filters.

Permission is granted by an admin.

• Stiki: The account must have either: (1) the rollback permission/right, (2) at least 1000 article edits (in the article namespace, not talk/user pages), or be approved after discussion with the developer.
• Huggle: Requires rollback permission in order to function but does not otherwise have an approvals system.
• AWB: Users must be added to a whitelist in order to use AWB. Only admins can edit the whitelist, and admins automatically have access. As a general rule, only users with more than 500 mainspace edits will be registered, and admins tend to only give access if a user has specified a task they want to use AWB for.

Discussion

Suggestion by Kudpung

• I'll start the ball rolling here with a fairly low threshold. Having seen plenty of the kind of errors that are made by editors who review AfC, I suggest that the minimum should be based on candidates having the choice of satisfying either of these two criteria (but not a lower mix of each).

1. Must have both Reviewer and Rollbacker rights, and have demonstrated that they have used these correctly within a minimum of 500 mainspace edits, and a minimum of one month tenure.

or

2. Must have patrolled at least 200 pages at WP:New pages patrol without recent error and demonstrated that they are a) familiar with the tags and deletion criteria offered by the Page Curation Toolbar without error. b) made significant use of the 'message to the creator' tool.

3. Above all, candidates must have demonstrated that they understand WP:PROD, WP:AfD, and WP:Notability, and are able to interact with other users in a polite, friendly, and helpful manner.

Kudpung กุดผึ้ง (talk) 02:25, 18 October 2013 (UTC)

Does Page Curation need to be singled out? Don't a lot of users do NPP with other tools, like Twinkle? I think you should refer to NPP-related tags and criteria in general. Also, I think reviewer/rollbacker would be fine with one-or-the-other, rather than needing both. Jackmcbarn (talk) 03:21, 18 October 2013 (UTC)

NPP should only be done nowadays with the Special:NewPagesFeed which does not use Twinkle. There may be a few editors still using the old page feed, but that system has been redundant now for a long time. Kudpung กุดผึ้ง (talk) 04:39, 18 October 2013 (UTC)

I happened to just ask Dragonfly67 on IRC the other day, and he doesn't use the curation tool... Not implying one way or the other whether or not he would want AfC reviewer (yes, I obviously realize as an admin it doesn't really make much difference), just wondering if someone like him that doesn't use page curation but has patrolled thousands of pages should really be excluded. TL;DR, I think that saying that curation tool use is a requirement of proper page patrolling is inaccurate. Technical 13 (talk) 04:49, 18 October 2013 (UTC)

Ditto... I patrol off WP:SCV and #wikipedia-en-spam. What we are looking for are speedy deletion accuracy and PROD/AFD nominations getting deleted. MER-C 05:19, 18 October 2013 (UTC)

MER-C, That's probably what you do, and naturally you are perfectly free to pick and choose how, what, and when you do, but patrollers should be aware of the recommendations at WP:NPP otherwise they are not really helping the project. I, for example, generally only look for blatant candidates for ultra speedy - and some of them I then summarily delete already - leaving the rest for other patrollers to figure out and learn from; I certainly don't plod systematically through the list, well, not these days - three years ago I cleared about 20,000 from the backlog in as many days, but I guess I was still full of Wikithusiasm. IMHO the new page feed and its curation tool is a brilliant piece of software; the only problem is that it's only any good in the hands of users who know what they are doing. Kudpung กุดผึ้ง (talk) 07:02, 18 October 2013 (UTC)

• I would like to see a certain number of articles created, perhaps 20, as a criterion for this user right. This would allow a fair assessment of the user's understanding with regard to article creation in my opinion. Additionally, I think the right could be bundled with autopatrolled just as well as sysop.—John Cline (talk) 03:23, 18 October 2013 (UTC)

While I like the idea of requiring some article creation experience I'm afraid 20 is far too high a threshold. I had been active here for 5 years and logged about 20,000 edits by the time I created my 20th article. Many of the most suitable candidates for AfC reviewer are those editors who have a lot of "wikignome"-editing experience - they generally don't create many articles. In any case "articles" is not a very useful unit of measurement - because both a 50-word stub about a village in Uzbekistan and a comprehensive GA-rated article about an obscure disease count as "1". I would give the right to the creator of one comprehensive article before I give it to a stub-mill with hundreds of three-sentence stubs, that just barely scrape past the minimum standards, on their scorecard. Roger (Dodger67) (talk) 06:47, 18 October 2013 (UTC)

Dodger67, for 'Autopatrolled' the default criterion is 50 articles. However, admins review these carefully, discount redirects and dab pages, and and don't generally accord the right to '100 stub wonders'. 1-line stubs about one specific topic area do not demonstrate a sufficiently broad knowledge of policies and guidelines, especially the mass creators who use AWB or their first stub as a template. Kudpung กุดผึ้ง (talk) 09:26, 18 October 2013 (UTC)

When I saw the topic, I was going to suggest article creation as a possible prerequisite. If as high as 20, I'd recommend that it replace one or more other requirements. If standalone, I would recommend a lower threshold, perhaps 5, but those articles must demonstrate knowledge of notability, reliability, independence, etc. If more than 5 articles are created, not all need to pass this criteria (some should could be stubs), but there must be 5 that do, and there must not be recent creations that demonstrate lack of knowledge in the critical areas. 78.26 (I'm no IP, talk to me!) 12:09, 18 October 2013 (UTC)

That largely echos my thinking. Article creation is good, but expansion of a stub to a full-blown article may be worth as much or more.--SPhilbrick(Talk) 13:40, 18 October 2013 (UTC)

While creating content within an existing article is an important measure, it presumes the existence of a notable article, rife for improvement. Article creation better demonstrates the all important ability of identifying notable subjects. AfD participation is perhaps another good way to gage clue in this regard.—John Cline (talk) 14:03, 18 October 2013 (UTC)

• Question as I'm seeing a lot of numbers or other rights being required here, which would be a major hurdle for many existing reviewers without of "grandfathering" of some kind. I'm not saying that these requirements are necessarily bad, just that they may be overzealous. Along the same lines as WP:CVUA for Rollbacker, I would like to think that a user without any of Reviewer or Rollbacker or 500 mainspace edits or 200 WP:New pages patrol or 20 created articles but who has demonstrated that they understand all of the proper policies (especially WP:Notability) via an AfC specific training program would be eligible. Technical 13 (talk) 03:35, 18 October 2013 (UTC)

• Kudpung said:

  Above all, candidates must have demonstrated that they understand WP:PROD, WP:AfD, and WP:Notability, and are able to interact with other users in a polite, friendly, and helpful manner.

You nailed it right there.

Any counts or other criteria we come up with are just ways to tell those who have done this from those who haven't, without having to spend hours wading through prospective reviewer's wiki-histories. davidwr/(talk)/(contribs) 00:47, 19 October 2013 (UTC)

@Kudpung: I hope you don't mind, I turned it into the {{tmbox}} at the top. davidwr/(talk)/(contribs) 01:08, 19 October 2013 (UTC)

I mind. Promoting one person's comment above all others with a big flashy spotlight is not conducive to consensus-building. -- ShinmaWa^(talk) 01:19, 19 October 2013 (UTC)

Suggestion by Anne Delong

• There was a lot of concern during the previous discussion that this would be a privilege which would be bestowed on some editors by others. This is how I envision the process working:

• The Afc helper script be changed to only function for those on the Wikiproject AfC reviewer list.
• The list could be on a protected page so that someone with regular reviewer rights would be needed to add names.
• On request, an editor would automatically be added if they had reached a certain level of editing (for example, 2000 edits and one year of experience).
• Editors wanting to review sooner or with less experience would have to meet the lower numbers of edits and time served mentioned above, and also convince a reviewer to add them to the list by demonstrating such items as Kudpung has mentioned above.
• Names could be removed if problems cropped up (such as frivolous or frequently incorrect reviews).

The reason I suggest the addition of an automatic pass level is that I believe that many of the people who supported the previous Rfc only did so because they believed that it would be an automatic rather than requested permission. —Anne Delong (talk) 03:50, 18 October 2013 (UTC)

• There were absolutely no mentions in the proposal on the previous RfC that the right would be automatically conferred. Criteria for the right, and how it would be granted were deliberately left open for further discussions. This discussion is to determine those criteria. What was clear on that RfC was the typical phenomenon on Wkipedia discussions that many people, especially those commenting later, do not fully read the preamble and proposition correctly and the following discussion and go off at half-tack - even introducing items that were expressly not required in the discussion.

I didn't mention it above, but but I would assume that current active reviewers who have not demonstrated any controversial issues with their reviewing would be grandfathered in.

I think requiring 20 article creations would be setting the bar too high. This is not required for NPP which has a similar need for knowledge of policies and practice but which does not require a permission (yet) and still suffers from the same problems: not enough patrollers, and often too little experience. I know I keep drawing these comparisons with NPP but I do feel it's relevant. Kudpung กุดผึ้ง (talk) 04:31, 18 October 2013 (UTC)

• Oppose criteria 2 New pages patrol is quite tedious and many users that could do it choose not to. I have to believe that asking people to do 300 NPPs will deter a ton of people from asking for AfC reviewer permission because there are plenty of other things most people would rather do than spend 20 hours doing NPP. Sven Manguard Wha? 06:15, 18 October 2013 (UTC)

I agree it is tedious. I did some quite some time ago and hated it. That said, it is eye-opening, and I wouldn't mind inclusion at a much lower level. --SPhilbrick(Talk) 13:44, 18 October 2013 (UTC)

• Given the exceedingly limited use of pending changes, what does reviewer actually signify? I agree with Sven's comments regarding NPP, as a long-standing patroller: 300 reviews is exceedingly high, both as something for the candidate to achieve and as something for anyone reviewing the candidacy to actually triage and check. Ironholds (talk) 06:37, 18 October 2013 (UTC)

Ironholds, I don't see anywhere that anyone has suggested 300 new page patrols. Perhaps if people would read discussions before they participate. I disagree most strongly that at NPP it is so difficult to attain a number of patrolls, I have done thousands and so have you. At the rate at which some patrollers review new articles, 200 patrols can be done in 200 minutes - alebeit probably as slipshod as some of the reviewing at AfC. Kudpung กุดผึ้ง (talk) 06:47, 18 October 2013 (UTC)

As usual, I'm going to ignore the pointless (and pointed) elements of your comment. To the remainder: the argument seems to boil down to "hey, you did it", which would be great if I wasn't the most active patroller for a solid two years by an order of magnitude I was even in a research study, how about that - "you can do it and I can do it" simply proves we can do it, not that it's achievable by anyone else. You know full well that our work on NPP is non-standard even for patrollers.

Sure, it's possible in a few hours, or days, or weeks if you actually want to put some effort in: that's not the point. It's a lot of work to put in to an activity you may actually have zero interest in - your interest is in AfC, not in NPP. It'd be like saying that for someone to be autopatrolled, they need to have extensive experience patrolling articles: sure, it's indicative of knowing what makes a good or bad article. It's also something that may bore them silly. I'm not entirely sure how excluding the people who don't enjoy NPP is going to help improve the quality or frequency of AfC work. I'd appreciate if you could address the reviewer comment as well. Ironholds (talk) 06:53, 18 October 2013 (UTC)

There are dozens of patrollers who have made 200 or more patrols - if you only do one a day that's about half a year, so please let's keep this in perspective. If the task is as thankless and boring as some have pointed out (which IMO it is), armed with that qualification they may find AfC more rewarding. No one is excluding those who have not done NPP - but you probably missed the alternative qualifications that were suggested. Whilst I see many parallels in the work of AfC and NPP, I see little or no correlation with PC reviewing. Kudpung กุดผึ้ง (talk) 07:18, 18 October 2013 (UTC)

Then I'm rather confused as to why you've recommended 'reviewer' as part of a qualification to get this right, since it's a PC-centric userright (unless someone can explain other uses it has, other than AFT5). Ironholds (talk) 07:24, 18 October 2013 (UTC)

Ironholds, it should be quite obvious that these are listed as examples of criteria for permissions that are accorded based on prior general experience and as incremental stages of user experience that demonstrate some metrics of knowledge of guidelines, policies, and practice for the purpose of access to different levels of meta tasks. We naturally have to start somewhere. You appear to be confused that we are discussing a MedWiki 'user right' according to the semantics of the Foundation, rather than a 'permission' as applied to this exercise. Kudpung กุดผึ้ง (talk) 08:46, 18 October 2013 (UTC)

Actually, it wasn't that obvious. I had to read it twice, after being initially puzzled that these were being designated as prerequisites. Then I realized they were examples of other rights, along with the criteria, so people could see example of criteria which could be used to think through the criteria for this right.--SPhilbrick(Talk) 13:49, 18 October 2013 (UTC)

Thanks, Sphilbrick. The text is indeed pretty confusing. Ironholds (talk) 16:40, 18 October 2013 (UTC)

• Support except for criterion 2 - but only because I have never done NPP at all. I've just thought of a way to imlplement "Criterion 3". Basically it ammounts to putting new reviewers "on probation". We use a mechanism similar to the "re-review" that is currently used as a "quality control" check during backlog elimination drives. Thus someone who meets the (deliberately low) technical threshold has their first reviews logged at a special page from where they are rechecked by experienced reviewers. The "probation" is lifted once the new reviewer has demonstrated comptence to the satisfation of the other reviewers.. Roger (Dodger67) (talk) 07:32, 18 October 2013 (UTC)

Dodger67, that's why aspiring candidates can choose between the two sets of criteria that fits their situation best. They don't need to satisfy them both. Essentially however, exactly what we are trying to do here is to avoid having to monitor the work of new reviewers as much as possible. This is currently being done on an ad hoc basis, but only when issues come to light. It would be impossible to do a double-control on all new reviewers - AfC resources are stretched too far already. Kudpung กุดผึ้ง (talk) 09:05, 18 October 2013 (UTC)

I think RogerDodger67 has the right mindset ... but I agree with Kudpung that manually reviewing past work, by having existing AfC folks manually monitor some please-check-me-for-accuracy queue, is not the way to go. Methinks the only approach that can put new reviewers on probation, and also automatically check their competence *without* requiring any additional effort from existing AfC folks, is to use an auto-test setup ... where the candidate AfC reviewer attempts to pass judgement on a stream of submissions, which some existing AfC folks have already judged. If the candidate gives the same answers as the existing folks, then the candidate has proven their worth. See my detailed suggestion-section, below. 74.192.84.101 (talk) 07:27, 19 October 2013 (UTC)

• Oppose rollback requirement. There are other ways to revert vandalism, for example by using Twinkle's rollback function. I find Twinkle's rollback feature superior to the standard rollback feature as it allows specifying an edit summary, and for that reason, I haven't even seen any need to apply for rollback permission on this project, although I occasionally use the rollback function on Commons. A user's choice to use alternative tools shouldn't affect the chances of becoming an AfC reviewer. --Stefan2 (talk) 13:11, 18 October 2013 (UTC)
• Oppose setting the bar wildly higher for WP:AFC than for WP:NPP as both largely compete for attention of the same volunteers. The requirements for both, while not identical, should be close. It's reasonable to ask that a reviewer be someone who has written an article or two which didn't get deleted, and understands the basic policies (particularly notability and sources), but set the bar arbitrarily high and the only result is to make an already-bad AFC backlog worse. That does no one any favours. K7L (talk) 13:43, 18 October 2013 (UTC)

Ironically there is no bar for NPP. That's why they have problems there too. I campaigned for years for a solution for the control of new articles which accumulated in the ill fated WP:ACTRIAL, and for improvement of NPP, and that was why we ultimately got the Page Curation system, but it still did not address the two issues: too few patrollers, and too little experience - and there is still an unacceptable backlog with some less easy articles not getting patrolled for months. Kudpung กุดผึ้ง (talk) 17:26, 18 October 2013 (UTC)

Suggestion by Sven Manguard

Despite "Consensus has been reached for this permission, this RfC is not for rediscussing whether a permission is needed." I still think that this is an incredibly bad idea and will ultimately complicate a process that is already so heavily bureaucratized and understaffed that it has had to come and beg people to help multiple times.

That being said, my suggestion is that rather than make the criterion based on vandalism fighting, we make it based on content creation. The permission would be given to:

• Autopatrolled (automatically, by making adding it to the autopatrolled package)
• Anyone with at least one GA or at least two DYKs
• Anyone that has a track record of positive work doing AfC reviews (before the RfC)

Admins should feel free to assign the permission to anyone that qualifies. Rather than set up a request board, the AfC instructions should instruct people looking for reviewer permission to ask an admin already involved in AfC.

Finally, and I can't stress this enough, the AfC reviewer userright group should never be used to determine recipients for mass messages. AfC has, in my opinion, a shockingly bad track record when it comes to soliciting participation from people that don't want to hear from AfC, and no matter what the ultimate decision about what the AfC reviewer criteria is, plenty of people are going to be given the userright despite having no interest in AfC reviewing (not least because admins will get the right automatically, as they do with almost every other right).

Cheers, Sven Manguard Wha? 06:32, 18 October 2013 (UTC)

• I'm still confused as to why we're talking about a userright. What technical privilege would it confer, and has anyone taken the time to ask the developers if this would actually be possible or even desirable as MediaWiki functionality? Ironholds (talk) 06:38, 18 October 2013 (UTC)

We can restrict use of the AfC tool. I'm not sure if that conforms to the definition of a userright. Re: "Has anyone taken the time..." Would you mind being a bit less combative? In the previous RfC, linked above, someone with (WMF) in their sig, who seemed to know technical stuff chimed in. --Anthonyhcole (talk · contribs · email) 07:03, 18 October 2013 (UTC)

Ahh. Me or Sven? Sorry if I'm coming off as combative; that's not the intent. Userrights are software-recognised things that permit or restrict MediaWiki actions; admin is a userright that allows access to things like Special:Block, autopatrolled lets MediaWiki automatically mark a page as reviewed, so on and so forth. From a MediaWiki point of view, AfC does not exist; it's not special functionality, just a set of pages. So I'm trying to ascertain if people have actually spoken to the developers and asked if this makes sense as a technically-implemented userright. If not, some of the comments above (rolling it in with autopatrolled, for example) seem unnecessary, and people might want to use less confusing terminology. Userright == MW-recognised status that grants access to special functionality. AfC is not software-recognised functionality.

The WMF-person I can see is Steven Walling; his statement was "I have no idea whether it will be even possible to fulfill the request from a technical perspective". So, this probably needs further investigation before rather than after criteria are established. Ironholds (talk) 07:08, 18 October 2013 (UTC)

Thanks for clarifying userright. We should probably determine what level of competence a person needs to have demonstrated before being permitted to review AfCs, before discussing whether we need to enforce it with a technical fix. I proposed earlier in this discussion that we use Wikipedia:Reviewer as a marker for adequate competence, and use social control to enforce it along with changing the AFC tool script to prevent anyone not on Special:ListUsers/reviewer from using the tool. But I pulled it, wondering whether that's setting the bar too low. Regardless, we can probably do what we want here without involving MediaWiki development. --Anthonyhcole (talk · contribs · email) 07:32, 18 October 2013 (UTC)

I don't know the plans for how to implement this, but a user right does not necessarily need to give access to extra special pages. For example, Commons has the OTRS-member, and the only difference between "OTRS-member" and "autopatrolled" is that an "OTRS-member" can add certain templates to a page without triggering Commons:Special:AbuseFilter/69. This user right could potentially be used in a similar abuse filter to prevent addition of certain AfC templates. --Stefan2 (talk) 13:17, 18 October 2013 (UTC)

(edit conflict)Once again, as there is little likelihood that a MedWiki solution will be granted - or even asked for, the question is moot. Some non MedWki methods have already been suggested and even by Brandon himself with whom I had a lengthy (and exceptionally friendly) discussion in Hong Kong. It's been mentioned dozens of times that permissions are needed for several MedWiki-independent actions. They are however listed at WP:PERM as the portal for permissions that are granted by admins. So again, we are discussing something that is not on the agenda of this RfC. Implementation/deployment comes later. Kudpung กุดผึ้ง (talk) 07:35, 18 October 2013 (UTC)

I am befuddled by this counsel! Considering notes 1 thru 3 of the original RfC, how can one say another's suggestion is moot upon its rendering? Otherwise, this is not a request for comment, but instead, a request for support; of ideas apparently already decided.—John Cline (talk) 09:35, 18 October 2013 (UTC)

Absolutely not, John. This is a think tank with an objective to define some criteria of experience for reviewing articles submitted to AfC. As stated in the previous RfC, what these criteria would be are for discussion (now here), and how it would be implemented will be discussed when the criteria have been established. One of the reasons that Wikipedia RfC fail or become overly convoluted is that there is often a tendency to discuss tangential issues at the same time, or ones that are not yet up for debate. Ironholds has made it perfectly clear that he will resist any suggestions to make this a MedWiki based 'user right', but has mistakenly assumed that that was the intention (on both this and the previous RfC). That said, if indeed any of the senior staff at the WMF decide that this 'permission' is of significant interest to entertain a MedWiki solution, we would be most pleased to hear about it, but we are not aware of any such offers as yet - in fact a closer look at my comments will reveal that I concur entirely with Ironholds that this is not a MedWiki operation, hence such suggestions are off topic as being evidently unworkable. Kudpung กุดผึ้ง (talk) 10:27, 18 October 2013 (UTC)

In my opinion it would greatly serve this discussion if there was a definitive answer regarding MediaWiki support. If the entire modification is to be implemented at the WikiProject level, then yes, we are straying off topic by suggesting a new userright, whether automatic or granted; and should therefore focus the eye of our brain storm locally. That said, the best solution to my eyes involves MediaWiki support, and I for one wish we had garnered that support already. Otherwise I think Graeme Bartlett is correct that a blacklist is the way to segregate bad apples and I suppose we could use discretionary sanctions to ban individual involvement where cause has been shown. Notwithstanding, I am optimistic that better ideas are forthcoming, provided we don't stifle the creative flow of ideas by the heavy hand of pessimism.—John Cline (talk) 13:24, 18 October 2013 (UTC)

Sorry to keep coming back to this John, but I think Ironholds (whichever hat he is wearing) has made it already abundantly clear. We're essentially discussing a set of criteria for a 'permission' rather than a 'User Right' per per Foundation semantics. I had an interesting in-depth discussion on this very topic with Brandon Harris, Erik Möller, and Steven Walling in Hong Kong and although they made some very interesting suggestions how we could approach an improvement to the AfC process, I do not believe there would be a spark of optimism for a MedWiki solution unless this were to have a cross-Wiki rollout. Personally I think it's best for us to find our own solutions locally. There is a faint chance that if they see we're making a superb effort in the right direction ourselves (as they did with NPP) they may step in towards the end bearing gifts, but I wouldn't bank on it. That said, although we want to avoid hat-collecting, I'm very suspicious that one of the reasons why NPP performs badly is that it ironically doesn't have a hat to collect although it demands far more knowledge than PC Reviewer or Rollbacker. Only today I came across a blatant long copyvio synthesis of multiple academic papers completely wrongly tagged by a 14-year-old patroller, who even apologised to the creator and removed the tag again! How many 100s of users would we need to blacklist before we have a few dozen reliable AfC reviewers left? A blacklist only shuts the barn door after the horse has bolted. Kudpung กุดผึ้ง (talk) 14:54, 18 October 2013 (UTC)

What were their suggestions? And this isn't a Foundation POV, this is a software POV - the two are very much distinct. Ironholds (talk) 16:39, 18 October 2013 (UTC)

Their suggestions are for a different discussion. As to WMF vs Software, you are best placed to know these things, but as far as the community is probably concerned, the Foundation holds the keys to development priorities, the human resources, the servers, and the funds. Please note that I support your theory that this is most unlikely to be accepted as a MedWiki request and I'm doing my best to stifle any sidetracking on the assumptions that it will. That said, from what I have heard from the Foundation staff and from competent programmers among the volunteer community, it won't be too difficult to find a local en.Wiki solution, whether a social one or one governed by some kind of script(s). Kudpung กุดผึ้ง (talk) 17:53, 18 October 2013 (UTC)

If all that we want from this is to limit who uses the Afc Helper script, I don't see why any WMF changes are needed. The script is developed by our volunteer coders here. As I mentioned in my suggestions above, to enforce this all that would be needed is (1) The list of reviewers on the "Participants" page would be protected so that someone with Reviewer status would be needed to add a username, and (2) The script would check the list and only work for a username on the list. Whatever criteria we decide to use, this combination should prevent random new users from coming along and adding themselves to the list. —Anne Delong (talk) 22:42, 18 October 2013 (UTC)

And what does that solve? The AfC Helper script is just a helper. There's no requirement to use it. It provides no functionality that a user can't do without it. AfC went a long time without having it, so I'm not sure what restricting it from some users accomplishes. If this whole RfC is about limiting a helper script, we really don't need an RfC at all. Just code it. However, the initial RfC made it very clear that this isn't just about the script. So, if it isn't a bit and it isn't just the script -- what is it? -- ShinmaWa^(talk) 23:42, 18 October 2013 (UTC)

I was just about to redirect this deep thread to ShinmaWa's question-section at the bottom. Agreed that there is no software-enforced requirement that only threshold-approved official AfC folks are permitted to perform AfC actions... but we can make that the *default* way that AfC is handled, and folks doing it *outside* the default way (with exceptions made to grandfather-in people with 10k edits that are using old-school tools or their own custom workflow or whatever) will therefore stick out. This makes it easier to see who is 'officially' doing AfC within the threshold-limits, of course... and if needed, we can tell people doing it *badly* outside the threshold helper-script world to please stop, right? I think enforcement without no cracks in the security is *not* the goal here, because WP:AGF. 74.192.84.101 (talk) 23:58, 18 October 2013 (UTC)

Suggestion by Lukeno94

• In my eyes, an AfC reviewer should experienced enough that they would easily qualify for the rollback tool. Putting that aside, I would agree that having at least 1000 mainspace edits is a good idea for an AfC reviewer. I would not say that the conventional reviewer right was enough; it's one thing reviewing and rejecting vandalism, and a whole other one reviewing a whole article. I don't see how GA/DYK/FA count should be relevant. The "autopatrolled" bar is too high for the AfC reviewer right; and as I've said before, you can be a great article writer but very poor at reviewing other's works. Having to patrol 200 things at NPP is excessive, although I agree that some experience is required (maybe 25-50?), due to that being one of the more relevant comparisons. I'll come up with my own proposal later, if I have time. Lukeno94 (tell Luke off here) 07:26, 18 October 2013 (UTC)

"Rollback" is a vandalism tool; it allows multiple edits to be undone quickly where a vandal has randomly hit multiYOUARESOGHEY LOLWTFOMGple articles. "Autopatrolled" is intended to keep extremely prolific but otherwise harmless new page authors from flooding WP:NPP. Neither necessarily infer a better AFC reviewer, although they normally are given to someone who is doing no harm. A good or featured article usually has multiple contributors instead of being WP:OWNed by one primary author; someone who'd submitted a pile of stubs in 2002 on valid topics, left the project for a decade and then returned to find some were expanded to GA/FA level would be given more credit than due. An editor which pulls a topic off the WP:AFD pile and rewrites it to WP:FA status, conversely, is not credited with creating an article. All of these metrics have their limitations - preview nothing before you save it and you can run up edit count more quickly, for instance. Experience writing valid articles or bringing existing articles up to some standard (off AfD to viable, off stub/start to B/A/GA, ordinary article to FA) is valuable but collecting privilege flags or edit count just for the sake of doing so does not always guarantee a better reviewer. K7L (talk) 14:18, 18 October 2013 (UTC)

Ideas by Graeme Bartlett

We should consider what we are trying to achieve here:

• Firstly we want to build the encyclopedia. So such a person should show that they can recognise the useful content for Wikipedia. The person should be able to understand what is and is not a suitable topic. They should be able to find a duplicte topic.
• Secondly we want to encourage the contributors, so we want the candidate to be able to talk to the contributors to explain what is needed to improve or to make an acceptable article. The person should be civil in their communication.
• Thirdly we want to keep it legal, so the candidate should be able to recognise a copyright infringement, or an attack page.
• Fourthly some nice to have features: The person can add categories and stub tags. The script seems entirely cabable of adding the almost useless orphan tag, so I hope our person can also show that they can edit articles to link to pages, including use of piped links.

Graeme Bartlett (talk) 07:59, 18 October 2013 (UTC)

Pretty accurately sums up what I said in my suggestion, Graeme. What we're looking for now are some metrics that define those qualities for the purpose of according the permission. Kudpung กุดผึ้ง (talk) 08:56, 18 October 2013 (UTC)

So I am not so demanding in predefined standards, but the candidate should be able to show these capabilities. If a person is asking for it they can show diffs that illustrate these capabilities. I do agree that NPP is quite a useful precursor experience for AfC reviewer. The other flags such as rollback and reviewer are not directly relevant, but certainly would show that the editor is constructive. If the person does not want to do 400 NPP items, perhaps they could do some apprenticeship work, perhaps checking AFC contributions and giving feedback to a mentor that would prove that they are on the right track. Graeme Bartlett (talk) 10:11, 18 October 2013 (UTC)

I see that 200 has grown to 300 and now to 400. Sounds a bit Falstaffian ;) Mine were but the first suggestions to get the ball rolling and I knew it would entrain some discontent; let's lurk awhile and see what others may suggest. Kudpung กุดผึ้ง (talk) 10:38, 18 October 2013 (UTC)

I like Graeme's ideas, and as a submitter I would love to work with an AfC person who met all these criteria... but I am very hesitant that some of them can be decided without fawning, interviewing, role-playing sessions, and other expensive overhead. (Yes, we are all volunteers for the most part, I'm talking about opportunity cost here... every minute an existing AfC person spends interviewing an AfC candidate, is *two* minutes that those people could have been actually whittling down the AfC queue backlog.)

   In particular, Graeme's point#2 about being an encouraging person, explaining things well, invariably civil, good looking, well dressed... okay, not those last two. But I hope the point is clear: there is no way to automatically test and verify those things. Just because somebody is good with those things in a one-hour interview is also no guarantee they will be that way *every* day, to more or less *every* contributor they happen to work with. Some people have a naturally sunny, cheerful, helpful disposition: I've met a few librarians like that, and many teachers. But for every one of those, I've interacted with hundreds if not thousands of fast-food clerks, waiters in restaurants, checkout clerks at the grocery store, floor assistants in retail stores, tech support folks via telephone or IM, and so on and so forth. It is *hard* to be consistently nice, consistently helpful, explain intricate details fully, and all that. Such people are diamonds in the rough, not grains of sand lying on the beach. If we *do* get a gemstone in AfC, I'd recommend we use them as a second-tier, for when contributors have trouble with their first tier person for whatever reason, they can be passed to the sunny cheery natural teacher sitting in the tier-two chair.

   Since point#2 took so much verbosity, I'll hit point#3 super-lightly: don't we have copyvio bots? And aren't BLP articles a specialist niche, given their legal-kryptonite-status, which ought be directed to *only* the AfC folks most experienced with such things? Point#1 methinks we *can* auto-test, see my 74-whatever comment below, and some of point#4 is also either auto-testable, or demonstrable in a three-minute (as opposed to three-hour) interview process. 74.192.84.101 (talk) 06:58, 19 October 2013 (UTC)

74 it sounds like you are raising the quality bar on the point 2, I was not expecting the behaviour always, but enough to do the job and encourage the contributors. The idea was not to have just gamers that can only push buttons. BLPs are most of what we have (may be companies too) so we need people to handle them too. But perhaps also we need people who can recognise their own limitations and not attempt something the mess up. So even someone that can decline a joke or vandalism can be useful if they just stick to that. Graeme Bartlett (talk) 21:33, 19 October 2013 (UTC)

Well... I'm not really trying to raise the bar, so much as point out that cheerfulness is a spectrum, but with some pretty well-defined focus-areas. I'm actually trying to lower the bar, if anything. I think we want the tier-one AfC candidates to be the equivalent of the sales-associates at the designer clothing store: efficient-n-quick with straightforward purchases, at least minimally friendly, but do not really have time (and thus do not really need to have the skillset) for solving difficult sticky-wicket cases. If you are trying to create an article about a BLP, who was formerly a relatively unknown business owner, but just announced their candidacy for the mayorship of a large city, and leapt to frontrunner status in the polls, then the sales-associate can send your article on through. If your little brother has a garage band, and the school newspaper mentioned their name once, and that is it so far, then the sales-associate can politely tell you WP:NOTNOW.

   The grey areas are more tricky, where something is borderline-Notable, but requires more depth in the sources, or whatever. I want those types of grey-area cases to be quickly glanced over by the first-tier sales-associate, and then passed back to the second-tier cheerful-librarian-associate. If the second-tier folks cannot solve the issues in a timely fashion, I want the third-tier to be, that the submitter is redirected to the WP:TEAHOUSE to find help doing the rewrite, and their AfC submission goes to the back of the queue. TLDR, rather than insist that our sales-associates aka AfC reviewers *must* be "interviewed for cheerfulness and tested on how sunny their disposition is", methinks we just need to remind everybody to be WP:NICE, which is required of *all* wikipedians anyways. If we happen to run across somebody that is *naturally* cheery and sunny, then we should then 'promote' them to tier-two work, where their special skill is extra-applicable: handling grey-areas. HTH. 74.192.84.101 (talk) 20:57, 4 November 2013 (UTC)

• We have copyvio bots, but they probably can't be used until AfC submissions are made on a namepage e.g; 'Draft', instead of a talk page or sub page. Kudpung กุดผึ้ง (talk) 02:59, 20 October 2013 (UTC)

• User:MadmanBot already scans AFCs (it misses at least some copyvios) when it's working. Patroller recognition of copyvios is still a must. MER-C 04:34, 21 October 2013 (UTC)

Okay... and how do we test whether an AfC reviewer-candidate possesses that special skill, ability to sniff WP:COPYVIO? There is a tool for analyzing whether URL#1 and URL#2 have copyvio problems. And there are bots that detect plagiarism, kinda-sorta. But short of glancing over the output of such tools, can humans really detect COPYVIO? I guess some things will be obvious, like a submission that says "copyright New York Times" at the top or the bottom of the text, or more subtly, text that has a bunch of internal links that are not wikilinks, but look like they came from a view-source-cut-n-paste job. But baretext submission, that was cut-n-pasted from the middle of some obscure website? Seems unlikely an AfC reviewer will detect the plagiarism with their spidey-senses. Maybe it's not that hard, because the plagiarized portion sticks out as a different 'voice' from the other portions of the AfC submission? (If there is a knack to copyvio-sniffing, methinks the parallel-primary-criteria scheme is useful training.) 74.192.84.101 (talk) 21:06, 4 November 2013 (UTC)

• agree -- phoebe / (talk to me) 19:18, 30 October 2013 (UTC)

Suggestion by Andy Dingley

Oppose any creation of a new right that resembles a "collectable hat" in any way, or that makes editors of this, "the encyclopedia that anyone can edit" more dependent upon bureaucracy.

We have a vandalism problem that ab initio editors can pop up, trash an article, post spam links and wander off. We have no checks on this. We allow unregistered editing and we allow unregistered editors to wreak all manner of havoc on established articles. I thus fail to see why we should start narrowing down AfC in particular to a subset of editors willing to jump through hoops.

In particular, making AfC review dependent upon a discretionary permission like rollback. I don't have rollback. I did have, and it was removed for a disagreement over regarding this edit / User_talk:Andy_Dingley/Archive_2009_September#Reversion as vandalism or not. Ever since then I've made a point of never asking for such a discretionary permission, lest it be pulled by some teenage admin with an axe to grind.

I can see some virtue to restricting AfC review (and think a lot more things, up to basic editing) should be restricted. But can we please keep this to a very lightweight, automatically-granted permission, not one dependent on cliques and fawning. Andy Dingley (talk) 09:39, 18 October 2013 (UTC)

To clarify a point raised off-line, there is a genuine concern that "editors who can't accurately review" shouldn't have this permission (that being it's point). We can attempt to judge this before granting it (which seems difficult to judge) but we can just as readily judge it after it has been awarded. If awarding the permission is a simple edit-count as a first filter, then it's easier to judge real skill by seeing some AfC reviews (and most editors just won't get involved anyway).Andy Dingley (talk) 11:20, 18 October 2013 (UTC)

On the matter of (pre-)judging-ability-to-accurately-review, see this section -- Wikipedia:WikiProject_Articles_for_creation/RfC_for_AfC_reviewer_permission_criteria#An_idea_from_Ross_Hill 74.192.84.101 (talk) 06:38, 19 October 2013 (UTC)

• I support these sentiments.—John Cline (talk) 09:43, 18 October 2013 (UTC)
• Support. AfC is desperately understaffed as it is, and unless the permission is automatic this already understaffed project will become a huge bottleneck for the encylopedia. At the very least, everyone who has previously done favourable work at AfC (10 or more good reviews) should have this permission from the outset. --LukeSurl ^{t c} 10:32, 18 October 2013 (UTC)
• Support this view. Any limitation on Articles for Creation will end up shutting out more good contributors than bad, and AfC is horrible backlogged already. Howicus (Did I mess up?) 14:16, 18 October 2013 (UTC)

Where are the hoards of volunteers 'without' a criteria who are already prepared to step in and review AfC submissions competently? AfC is indeed desperately understaffed as it is, and does not have the person-power to review every reviewer's work. That would only make the bottleneck worse. Some are obviously getting it wrong and yet others blatantly abuse the system for their own ends. We either want reviewers or we don't, but appointing them through some arbitrary automated selection method without any real control would probably lead to greater disaster. The permission has been created by consensus. This is an RfC to determine the criteria for that permission and not to re-debate the need for it. Once the criteria have been established, it will be further discussed how to implement them specifically in a way that it does not become a trophy for the hat-collectors, with as little 'cliques and fawning' as possible, and avoid being pulled by the (fortunately) ever dwindling corps of teenage admins with an axe to grind. Kudpung กุดผึ้ง (talk) 11:03, 18 October 2013 (UTC)

I think getting more AfC candidates is a job for WP:RETENTION, and similar anti-WP:BITE organizations like the teahouse. Bluntly, it is very difficult to *increase* the percentage of editors that will want to get involved with AfC work, by demanding they first meet some threshold-criteria. (That is not strictly the case, which is why I said 'very difficult' and not flat impossible... one could imagine threshold-criteria like 'willing to accept USD$100/hour from wikimedia foundation for their AfC work' that would *dramatically* increase the pool of editors willing to fight for an AfC slot, but as a class those tend to be unrealistic).

   I think what Andy and LukeSurl are trying to say is that the point of the threshold-criteria is to keep from accidentally reducing the number of AfC candidates *too* much, while still satisfying the basic goal that the threshold-criteria gives us a usable metric from separating the wheat from the chaff. We want the threshold to prevent COUNTERPRODUCTIVE folks from becoming AfC workers, where their net contribution is negative, because they make so many mistakes which other folks end up needing to clean up later on. But if we require fawning, or non-automatic AfC-permbit acquisition, or tons of paperwork, or running the gauntlet ("in order to get the AfC-permbit you must undergo RfA -- even if you already have the admin-bit"), or significant friction-slash-overhead, we shoot ourselves in the foot. Too much friction, and the overall benefit of having a threshold (eliminating N counterproductive candidates) will not outstrip the overall disadvantage of having a threshold (eliminating M productive candidates!). Agreed that we don't want an "arbitrary automated selection method without any real control" ... but we do need it automated, preferably non-arbitrary, and with as little bureaucratic friction as possible, both to keep from tying up existing AfC folks in resume-review-and-interview stuff, plus also to keep from tying up AfC candidates in fawning-and-paperwork. 74.192.84.101 (talk) 06:35, 19 October 2013 (UTC)

I think we do need a human assessment rather than an automated one. Perhaps it can be easy to get but then easy to remove if there are stuff ups. Perhaps a list such as for AWB can be useful, and alternative could be that we just have a black list. The kind of hat that people would not want to collect is a possible. The hat could be "restricted from AFC review" and only stop people from doing it. We could have other bits for vandals or clueless or copyright infringers. Then these are the people that don't get to operate it. for the axegrinders, we need an axeginder bit too! Though I think I am stretching this to non-seriousness here. Graeme Bartlett (talk) 11:21, 18 October 2013 (UTC)

The Eric Corbett CIVILity-inapplicable bit. Andy Dingley (talk) 11:25, 18 October 2013 (UTC)

For the axegrinders we need an angle-grinder. Kudpung กุดผึ้ง (talk) 11:40, 18 October 2013 (UTC)

An idea from Ross Hill

I agree with all the goals put forward by Graeme Bartlett.

So far the suggestions have mentioned edit count, rollback, AWC, etc. as prerequisites. However none of those things truly show that one can review articles well. What is a better way to prove your worth at reviewing articles, than reviewing articles? I propose that every candidate find 5 pending articles they would decline, and 5 they would accept and they would have to explain their reasoning, citing policy. They should also be able to explain WP:BLP, WP:NOTABILITY, WP:VERIFIABILITY, etc. An admin would then review their responses and choose whether to accept them as a reviewer. Thoughts? Ross Hill (talk) 16:30, 18 Oct 2013 (UTC) 16:30, 18 October 2013 (UTC)

It's a bit hard to find acceptable articles...how about 5-8 articles total, whether acceptable or not? Howicus (Did I mess up?) 23:46, 18 October 2013 (UTC)

Oppose "admin would then review". I don't see any reason why administrators are required for this process, unless there's a technical implementation requirement for them to be. -- ShinmaWa^(talk) 23:48, 18 October 2013 (UTC)

I wasn't really focussing on any specifics. I don't care if it's an admin, or an experienced reviewer. 5 articles, or 3. I just want feedback on the idea. Ross Hill (talk) 23:52, 18 Oct 2013 (UTC) 23:52, 18 October 2013 (UTC)

Fair enough. Of course, how feasible this suggestion is depends upon the implementation (which is why it is folly to attempt to separate criteria from implementation). However, given all the social-based implementation ideas presented so far, including AfC mentoring, elaborate testing, and the like, this one is the best so far, I think. However, it would need to be fleshed out quite a bit on the specifics of who gets to review, based on what objective criteria, and -- to beat the dead horse -- how approving an applicant would be implemented. -- ShinmaWa^(talk) 00:03, 19 October 2013 (UTC)

I have fleshed out an implementation plan for Ross Hill's idea, which I believe is the only truly fair (and predictive) basis for 'testing'. (My answers to ShinmaWa's questions are nobody, based on the objective performance of existing trusted AfC folks, and automatically based on the specified X-and-Y values at the time -- or perhaps retroactively.) The other criteria being discussed (editcount/etc) are all secondary criteria, which might be useful as a way to pass-the-test-without-testing, but cannot replace the trial by fire of AfC work itself. Rather than choosing articles at random, and let possibly-biased editors make the call on whether the candidate judged correctly, I suggest using real articles that are really going through AfC. If the candidate gets right answers (where 'right' is defined by the answers the actual AfC person gave) on enough of the articles, they too become an official AfC person. See here -- Wikipedia:WikiProject_Articles_for_creation/RfC_for_AfC_reviewer_permission_criteria#Suggestion_by_74-whatever

support this. If we must have a 'crat allocated privilege, then at least let's bind it to the real task in hand. Candidates review some (clarification needed) unreviewed AfC candidates, of which at least a couple must be judged pass/fail as a result. Andy Dingley (talk) 19:26, 19 October 2013 (UTC)

Suggestion from Sj

It's distracting to separate criteria from implementation here, since they are closely tied together. (The criteria you use affect how you can implement it, and v-v.) So here's a joint suggestion:

Simple option

Maintain a list of reviewers on a wikipage. Let anyone add themselves; remove those who aren't working out yet. Add a feature to one of the popular review-tools that checks to see if new articles on an AfC topic are created by users who aren't on the list -- a flag that someone else should doublecheck the work.

Tying this right to 'edit count' or 'rollback' seems like a terrible idea to me. The number of people willing and able to do this work is tiny; you can interact with them all personally. Instead, tie it to a single back-and-forth welcoming interest and asking people if they feel comfortable they know what a good article looks like [with pointers].

Future technical option

Combine this with the Reviewer flag. Make this the Flagged-Revs workflow for the very first rev of an article. Make it something that is given automatically to people meeting certain threshholds, and to anyone else who asks. Allow it to be removed for misuse; but most granting of the right should be automatic, other than time spent welcoming new collaborators.

The flag should allow access to tools that make AfC work streamlined and easy, and that update any special pages that track requested articles. (In comparison: anyone, with or without this right, can browse the AfC requests and create articles based on them. But it won't be checked off of the queue until a reviewer checks that work.)

Aside: it seems to me that the impact of the Reviewer flag has been weakened by the requirement that admins apply the right, with no automatic way to get the flag. This is unlike basically every other reputation-ladder I know of. Our lack of automatic activity-based rights (other than autoconfirmation) is a waste of energy, and seems self-perpetuating. – SJ + 17:14, 18 October 2013 (UTC)

I totally agree. I am still against a user right for this, and reading over what is written here, it looks like what we need is a "reviewer block" for bad reviewers, not an extra reviewer right. All of the criteria I have read above just reinforces my scepticism, because whenever someone talks about "grandfathering in" they really mean "let's keep this cabal small, trusted and among ourselves". We really need to start trusting newbies again like we did back in 2006, or the editor retention rate is going to drop more and more rapidly as the "grandfathers" start to drop off, for whatever reason. Jane (talk) 10:16, 28 October 2013 (UTC)

Suggestion for Buffbills7701

Alright, I'm going to cut right to the chase. I think that in order to get the reviewer right, you need to get past this. The mentoring program is currently a work in progress, but when it's done, it would be the perfect solution for the new AFC reviewer right. buffbills7701 20:29, 18 October 2013 (UTC)

This should be one way to get the access to the AFC Helper Script. It should not be the only way (e.g. most very experienced editors with good reputations shouldn't have to "go to school" to get access to the tool). davidwr/(talk)/(contribs) 23:49, 18 October 2013 (UTC)

Seems like a great way to open a hatshop for the currently inexperienced. No, or at least very few, long term editors would go near it. Andy Dingley (talk) 00:40, 19 October 2013 (UTC)

Andy, I'm sure Buffbills meant 'one way' as David expanded. Every single user right from Confirmed through to Bureaucrat is a millinery in our information mall, but generally only on the lower floors - anyone who has worked extensively at WP:PERM and WP:RfA knows this. However, the systems of scrutiny that accord those rights generally function well but there will always be a tiny few who loose their flags - especially admins who have an axe to grind. At the lower levels, it is even more rare for PC reviwers, Rollbackers, File Movers, Autopatrolled, etc. to be demoted, but it does happen. I've never been subject to sanctions, but as one who was bullied by two teenage admins early in my Wiki career, and completely bullied away by an admin (now desyoped) from one topic area, never to return, where I had most to offer the encyclopedia, I do follow the ANI/AN, RfC/U, and Arbcom rituals very closely even if I don't participate much there. As an admin however, I don't have any axes to grind.

Let's not get too uptight or paranoid about the occasional hat-collector slipping through the net, a system of control over who can process AfC submissions is far better than none at all or one that is accorded automatically based simply on editcount/tenure, etc. Possibly those who work regularly at AfC and its maintenance are more aware of the issues than those who don't, but what we are here to do is ask the broader community for their opinion on, and to suggest, a set of criteria that would largely contribute to improving the quality of AfC reviewing, ensure that all reviewers are singing from the same page, and are friendly to the the submitters. The permission does not grant any further rights or hamper the work of article creators who know what is expected from an article that will survive legitimately in the encyclopedia. Kudpung กุดผึ้ง (talk) 04:47, 19 October 2013 (UTC)

I would agree with David here. We have a need to solve the current problem of poor quality reviewing. I believe we need this right, but it should be very low, based on a simple mileage count – then if needs be, withdrawn from poor reviewers, based on the quality of their reviews. Secondly we can achieve this by encouraging experienced editors to take more part (AfC review is not rocket science) and anything that could be seen as "patronising" is hardly likely to achieve that. How many 5+ year / 10s of kedits editors want to be "mentored" by someone who has maybe 6 months of springy-tailed editing inbetween school? I spent a chunk of last week being lectured on 1950s motor racing history by someone who's barely old enough to have a driving licence, but here they have the free time to do a lot of typing, so they get to shout loudly and often. Andy Dingley (talk) 10:15, 19 October 2013 (UTC)

Buffbills7701, editors with only double digit edits regularly add their names to WP:WPAFC/P list and due to the immense workload we're not always quick enough to do something about it. Last week one registered with the sole purpose of passing their own articles. One of our concerns therefore is for the grey area of editors who review, but whom we are not aware of. I support the idea of a school for aspiring reviewers and I'm currently working with other editors to set one up. I don't believe genuine hat-collectors are very interested in going through the rigours of our various training systems (I completely redesigned the WP:CVUA from the ground up and also set up an NPP school) . One of our standard answers at WP:PERM (Rollbacker) is "Hi, I appreciate your enthusiasm but with only 46 edits to mainspace I don't think you have sufficient experience yet. When you have made at least 200 edits, you may wish to enroll at the Wikipedia:Counter-Vandalism Unit/Academy to learn more about it." Kudpung กุดผึ้ง (talk) 04:47, 19 October 2013 (UTC)

Here's what I don't understand about all this. You've brought up this editor who registered an account to "pass his own articles" on a number of occasions. So. What. You act as if this is some kind of real crime against the project. In reality is that once he's a registered user, he has every right to create his own articles in mainspace as much as every other user does. If he wants to clear the duplicate article out of AFC in the process, there might have been better ways to do it, but overall, he didn't hurt the project at all and he certainly didn't hurt AFC one bit in doing so. That argument is completely a red herring and I do wish you'd stop using it. -- ShinmaWa^(talk) 19:06, 3 November 2013 (UTC)

Suggestion by 74-whatever

Kudpung and Anne and others have suggested various secondary criteria for the threshold: edit-count, NPP, and so on. Lukeno pointed out that some secondary criteria (like participation bringing something to GA status) have little relevance, because most of the AfC stuff is nowhere *near* that status. Several people have pointed out an automatic-grant-the-bit solution is the best way to minimize bureaucracy, but other people have countered that the human element is crucial, for most secondary characteristics do not really tell us if the candidate will be any good at judging AfC submissions. It is important that they be good at this task, because too many false-negatives will cause a dramatic amount of work downstream, and of course a lot of drama if an 'accepted' article is then speedy-AfD'd the following week. I believe there is a way AfC folks can have their cake and eat it too. We should judge the worth of potential AfC folks, based on how they would do on real-world AfC submissions, compared to current AfC folks on those same submissions.

• Candidates wishing to get the authorized-for-AfC bit test their skills against real-world AfC submissions
• Threshold should be an X% success rate on a minimum of Y real-world AfC decisions
• In parallel, and blind to such candidates, the already-authorized AfC folks do their normal work

• Example test: on Wednesday evening, Anne Delong judges ten AfC submissions from the queue; I do the same, without seeing any of her decisions
• Example math: Anne's answers were yyNNyyNNyy to those ten, and my answers were yyNNyyNNNN , which means I made two mistakes (Anne is perfect -- good work Anne :-)
• Example fail: if the threshold chosen is X>=90% and Y>=10_decisions, I satisfied my_Y>=10 but I failed to satisfy my_X>=90.
• Example learn: determined to get there, I study Anne's answers (now visible to me after my test-session), and keep trying.
• Example win: in my next test-session, I judge ten more of Anne's cases in parallel, and make no mistakes. Now my_Y=20 and my_X=18/20, which means I just auto-passed with my_Y>=10 and my_X>=90%.

Disadvantages:

• the test-session itself is duplication of 'real' work (Anne is working -- I'm only *simulating* work she already did)
• somewhat difficult to explain the concept of auto-testing in parallel (cf verbosity of this proposal)
• may be *quite* difficult to implement the concept of auto-testing in parallel, since Q&A with the submitter is not something we can simulate
• likely impossible slash infeasible to really make the 'blindness' of the auto-test secure (if Anne emails me the answers I *will* pass)
• even if we posit that security is not a big deal, and Q&A can be elided, still need a dev to write some code for auto-testing (not true of e.g. simplistic editCount>=1000 threshold or similar)
• hard to pick the initial Y ... make it too high, and nobody will try, make it too low, nobody will fail
• hard to pick the initial X ... make it too high, and *existing* AfC folks will be eliminated, unless grandfathered in
• just because you crammed, and memorized the policies long enough to pass a test-session, does not mean you really are good at AfC later on
  • "Kudpung: Last week one registered with the sole purpose of passing their own articles." Somebody could cram for the test-session with that purpose in mind, too. Only an admin can catch that.
• scoring well on the auto-test does *not* necessarily make you a good AfC judge... it depends on whose answers you correctly mimic'd!

Advantages:

• threshold is real-world *primary* criteria, not secondary
• although no humans are involved with granting the bit once I pass, a real human is doing the testing (Anne is testing me)
• as with anything in wikipedia, WP:IAR means that even if I auto-pass, some admin can always *manually* remove my authorized-bit later on
• fair nature of the automated testing means no complaints about bias/fawning/etc
• easy to auto-grant the permbit when the threshold is met , with a database table of people-who-passed-the-automated-testing
• easy to auto-warn an 'official' AfC person when their ongoing work falls below the testing-threshold at any point
• easy to retroactively adjust the threshold-values of X and Y upwards to improve quality, or downwards to improve reserve-troop-strength
• difficult to explain 'on paper' but in practice easy to explain... watch what Anne does today, tomorrow do what Anne does, if you do well you pass, if you don't you can try again.

Hope this helps. 74.192.84.101 (talk) 23:00, 18 October 2013 (UTC)

p.s. Forgot to mention that I agree that *some* sorts of work should automatically be given the AfC-permbit. Have three years and 10k edits with no blocks in the past year? You get the AfC-permbit without needing to pass the X-out-of-Y-auto-testing-threshold. 42 edits on enWiki, but 10k edits on deWiki? Prolly you have to take the auto-test, as a real-world check on your ESL skill. Have 333 NPP credits? Ditto. Have 10 new articles in existence, each older than a month without being deleted? Ditto. Member of arbcom, passed an RfA (regardless of whether you still hold the admin-bit), surname Wales? Ditto ditto ditto. But these secondary criteria should be, well, secondary. What matters is not your edit-count, but how your judgement matches up against Anne Delong's judgement. Our existing AfC personnel should be the gold standard, both now, and five years from now. Auto-testing is a self-reinforcing metric of 'goodness' methinks. 74.192.84.101 (talk) 23:17, 18 October 2013 (UTC)

And in particular, one group that should automatically get the AfC-permbit was mentioned by Anthonyhcole (besides the 1500 admins), namely, the 6000 people with the Reviewer-permbit. Much like I'm suggesting here, there is a trial period. However, the threshold-criteria for Reviewer-permbits are not numerical and automatic, but require an interview process: knowledge of the reviewing-guide & vandalism-policy, familiarity with WP:COPYVIO / WP:BLP / WP:NPOV / WP:OR / WP:V / WP:NOT, and finally "have an account with track-record of routine editing". 74.192.84.101 (talk) 07:14, 19 October 2013 (UTC)

The above suggestion that at first the new reviewer would review "in parallel" seems overly complicated, but there would be a simple way to implement this. A new reviewer could pick out a submission to review, and instead of actually reviewing it, leave a message on the Afc talk page saying something like "I think that XXX is ready to be accepted" or "I think that XXX should be declined with this decline reason ___ and I would leave this message:___". Then any of the regular reviewers could say "Looks good to me, go ahead". That way we'd all be "mentors" and the new reviewer would safely get practice. —Anne Delong (talk) 23:24, 19 October 2013 (UTC)

Anne, similar ideas have been posited above in other sections. I personally do not support any solutions that will: eiher increase the workload of other reviewers or project editors working at AfC, and/or slow down the reviewing procerss; What this RfC asks for is not alternative solutions, but for a set of criteria of experience. Although the rights Rollbacker, reviewer, template editor, File mover, etc., may in some instances not be a good parallel, thier granting system is not dependent on any form of probation or monitoring of their progress. I think we need to look for a similar, simple 'granting' process here based on experience than can be quickly investigated (edit count, type of edits, talk page comments, block logs, etc.,) rather than look towards implementing a more complex and time consuming process. An AfC Academy has now been developd and any aspiring reviewers who fall short of the criteria that we will set here can be referred to that for training if they are serious about becoming reviewers in much the same way as we have a CVU school and an NPP school - bearing in mind that this latter is generally only used when NPPers (who don't need any quals at all) persistently get their patrolling wrong and are asked by an admin to stop. Kudpung กุดผึ้ง (talk) 03:18, 20 October 2013 (UTC)

I've already given my opinion about the qualifications above, but you didn't like that either, so I will go back to reviewing now. —Anne Delong (talk) 03:24, 20 October 2013 (UTC)

Hello again Anne, thanks for the comments. Yes, my suggestion is obviously quite complex, to understand and to implement, whereas your mentor-by-humans approach is straightforward and easy to implement. But the worry for Kudpung is that you and the other AfC regulars are *already* overloaded, so mentor-by-humans is going to pull expert AfC reviewers into mentoring (and thus necessarily out of AfC work), and I share that worry. My complex review-in-parallel scheme is designed to let the computer be the mentor, so that a beginning AfC candidate can test their mettle against your known skill, *without* you needing to directly mentor them. Once the top candidates were known, then mentoring would be the next phase. Anyways, as Kudpung points out, my solution is not what this RfC is for... this RfC is for coming up with a bunch of secondary criteria, that can be used for autogranting the AfC kinda-sorta-like-a-permbit-yet-not-really. (My scheme attempts to dispense with secondary criteria, and directly measure How Good The Candidate Is At AfC Work Itself.) Appreciate the criticism, danke. 74.192.84.101 (talk) 20:38, 4 November 2013 (UTC)

Question from ShinmaWa

While this RfC is primarily about the criteria, which I fully recognise, some thought into implementation needs to take place lest we paint ourselves into a corner that can't be implemented. A lot of discussion is about a UserRight bit, which has technical issues which Ironholds discussed above. There's also been a lot about restricting scripts and tools. However, while there are a number of scripts and tools available to assist with AfC, they are 100% optional. Everything done at AfC can be done without a single tool in place and was for a very, very long time.

When boiled down to its essence, AfC requires that users be able to 1) Move pages from the "Wikipedia talk" namespace into article space and/or create new pages in article space and 2) Edit existing pages in the Wikipedia namespace. That's it. Every autoconfirmed user on the planet has the capability to do this. Restricting the tooling will just restrict the tooling. It won't actually keep a single user from participating in AfC.

So, the question is this: For this criteria/permission/etc to be meaningful, failure to meet this criteria somehow has to prevent users from creating articles nominated in AfC (which has all kinds of bad second-order impacts) and prevent users from editing articles in the "Wikipedia talk" namespace that "belong" to AfC (ditto). Just how are we going to go about this? -- ShinmaWa^(talk) 23:15, 18 October 2013 (UTC)

"if it isn't a bit and it isn't just the script what is it?" From what I understand, it is a community standard, used by enWiki, to see who is 'qualified' to be an AfC person. It is like the ISO standard for papersizes, where there are tolerances plus-and-minus a few micrometers, but if you are within the tolerances you can say you are ISO-standard-sized A4 paper, or whatever. That does not mean that *every* piece of paper is ISO-standard, nor even that ISO-standardized paper is the best (arguably vellum is the best). It just means that, if you have satisfied whatever threshold this discussion ends up recommending, that you become a Recognized Official AfC Member In Good Standing, subject of course to other admin-actions that might keep you from acting on your over-the-threshold qualifications. Maybe someday it will be a 'real' permbit like the admin bit, where security matters... for at present, methinks it is just metaphorically an AfC-permbit, loosely enforced by community standards rather than strictly enforced by software. 74.192.84.101 (talk) 00:03, 19 October 2013 (UTC)

"For this criteria/permission/etc to be meaningful, failure to meet this criteria somehow has to prevent users from creating articles nominated in AfC" (emphasis added). I think the 'somehow' is going to be, by manual admin intervention. If you have not satisfied the threshold, and you keep submitting perfect articles as AfC, which always pass with flying colors, who cares? If you have not satisfied the threshold, and you 'manually' create articles without the AfC helper-script, sooner or later an admin will make it their business to care, and call you out for disruption. 74.192.84.101 (talk) 00:06, 19 October 2013 (UTC)

So, a completely social-based implementation. How does criteria play into this then? This approach winds up being a no-op and bringing us right back to where we started. Specifically, "If you don't meet our criteria, you can't play in our sandbox" just becomes "If you are being disruptive, an admin will intervene". However, that's already the case. We don't need an RfC or a bit or criteria or any of that to have admins deal with disruptive users. So, I'm quite confused as to what that accomplishes realistically. -- ShinmaWa^(talk) 00:16, 19 October 2013 (UTC)

Those who do not meet the criteria will be slowed down significantly, and those who make mistakes and are warned and continue to make mistakes can rightfully be called disruptive. Likewise, those who have been given access to the tool and had it later revoked and who come back and "do it by hand" in a substandard way can also rightfully be called disruptive. Wikipedia already had mechanisms for dealing with disruptive editors. Revoking access to the tools for editors who are merely incompetent can slow them down enough to encourage them to think about what they are doing, which will hopefully mean they will have a higher rate of competent reviews. Let's suppose Joe Novice Wikipedian is trying to help out and somehow gets access to the tools and makes 30 reviews in 2 days, but botches half of them. He gets access to the tools revoked but he is determined to help out. Over the next 2 days he does only 10 reviews because he's been slowed down for lack of access to the tools. At worst, we have 10 reviews to re-review and 5 to clean up. But hopefully he'll be more accurate becuase he's working slower (and gaining experience as he goes) and only flub 2 or 3. davidwr/(talk)/(contribs) 00:42, 19 October 2013 (UTC)

I can certainly respect that. In fact, I even support this approach. It doesn't take the gun away, but it removes the fully automatic selector switch. There's certainly precedence for this with Twinkle and the like. However, this begs the question if this actually meets the consensus formed in the original RfC. While I opposed that RfC, many people didn't, and I suspect some supporters might see restricting just the script as being a half-measure. *shrug* Thanks for responding, davidwr. -- ShinmaWa^(talk) 01:10, 19 October 2013 (UTC)

There are some very relevant comments in this thread, and I'll just reiterate that the consensus in the previous RfC was There is community consensus for the introduction of a requestable permission which will be required to review articles at Articles for Creation. - nothing more, nothing less, and that is what was asked for. Firstly, I believe even a half-measure is better than none at all, to the exclusion of any arbitrary automated granting of the access. Having a list of users who are 'authorised' to use the script is also a kind of 'half way' that we already have, but as I mentioned somewhere above, we need to get all reviewers on a list. Naturally if they get their flag removed, under the current technological aspect of the process, there is nothing to stop them continuing to do manual reviews; it would certainly slow them down, but we would know who they are. Secondly however, with a couple of thousand submissions in the queue, we don't know who is actually doing the reviews at all - we just don't have the person-power to do a double check on every submission that gets declined, moved to mainspace, or CSDd under an appropriate criterion. But we are diverging here - we need to set the criteria for permission first - and that shouldn't really be too difficult (we have enough examples cited above for the granting of various user 'rights' that do need official WMF approval) , then see how they can be technically or socially implemented. 06:06, 19 October 2013 (UTC)Kudpung กุดผึ้ง (talk)

As mentioned by me and several other people, it is folly to attempt to separate the criteria from the implementation and it is awkward to attempt discuss one without the other. One impacts the other at a fundamental level. Further, many of the suggestions demand a certain implementation and/or precludes others. So, if we can't talk about implementation, then our criteria options become severely limited to stuff like edit counts and other similar statistics. In essence, the "no implementation" restriction steers down a very narrow set of options -- namely the options that you suggested at the top at the RfC. I think we need to look beyond that scope. -- ShinmaWa^(talk) 02:55, 20 October 2013 (UTC)

What Shinmawa raises is a fundamental flaw, this RfC is invalid

A userright represents the ability to use a technical feature. Since this userright won't place any technical restrictions on anyone, there is no point at all in creating it. Access to the common scripts could be toggled with or without a userright. But they could still load the same exact script via their custom JS interface, and we can't stop them from doing that. So in my mind, the prior consensus is irrelevant because it is not possible to implement.

If there is a desire to create a socially enforced white or blacklist, then we should be talking about creating a process for that, and this RfC should be closed and reframed properly, with first a discussion about whether it should be a whitelist or a blacklist, before any criteria are proposed.

A blacklist makes the most sense to me, because there is nothing at all stopping someone who isn't whitelisted from processing AfCs, and if they do it correctly, are we going to really block them for failing to participate in the bureaucracy? Gigs (talk) 18:31, 28 October 2013 (UTC)

Suggestion by Davidwr - Leave access to "AFC Comment" unrestricted

Leave the "Comment" button on for everyone by default.

If an editor abuses it, they can be blacklisted.

The kinds of comments editors leave are probably the best judge of whether they should get access to the rest of the buttons. davidwr/(talk)/(contribs) 00:31, 19 October 2013 (UTC)

I do not think that new people will bother activating the tool jsut to get the comment button. However I don't think that being able to comment is harmful. After all it is not that hard to edit the article and add a comment, even using the correct afc comment template is not that hard to do. Blacklisting against adding comments I suspect would be about equivalent to a topic ban. Since it is so easy to bypass I would not suggest implementing a comment blacklist. Graeme Bartlett (talk) 21:01, 19 October 2013 (UTC)

Suggestion by Davidwr - Make it a "throttle" like AccountCreator

Tweak the AFC Helper Script so everyone can use the full set of tools on no more than a handful of different submissions in a rolling 2-3 day period.

Those who abuse the tools can be blacklisted. davidwr/(talk)/(contribs) 00:31, 19 October 2013 (UTC)

I assume that you do not want to rate limit it for every one, so the users with the permission can review faster. Although I suspect our bulk and speedy reviewers do make some errors too, such as we can see by the number of AFDs and prods that pop up. Also the stuff declined for a weak reason will not show as a problem that way, but just drive away contributors and content. However I do like the idea to do a rate limit for the people with no permission, but then also add their work to a special list for extra review by others. Graeme Bartlett (talk) 20:54, 19 October 2013 (UTC)

Suggestion by Davidwr - many routes to full access to the tools

There shouldn't just be one route to get access to the full AFC Helper Script.

I'd give access to these buttons to anyone requesting them who:

• is a long-time Wikipedia editor with no recent relevant problems
• is grandfathered in because of significant recent AFC participation and no recent relevant problems
• demonstrated competence through intelligent, accurate AFC comments or direct feedback to editors over an extended period of time and a significant number of articles
• demonstrates competence through intelligent collaborative content-improvement in other areas of Wikipedia over an extended period of time and over a significant number of articles
• is under the training or sponsorship of another experienced AFC editor, editors, "acadamy," or similar, or has been declared to be competent to have the tools by their sponsor
• while not clearly making the cut on any one of the above, goes through a short (1-3 days?) discussion period and get the rights if there is a consensus to give it to them.

Revocation should be relatively easy, with the typical "appeal" taking the form of the 1-3 day discussion period outlined above.

Even after adoption, this list should not be cast in stone. davidwr/(talk)/(contribs) 00:31, 19 October 2013 (UTC)

This sounds like a reasonable idea. It is better than a hard list of requirements that may seriously limit the numbers of new reviewers. Graeme Bartlett (talk) 20:49, 19 October 2013 (UTC)

I particularly like davidwr's mentorship idea. When I started reviewing I hardly knew how to do anything, so I just started out asking questions and reporting problems on the Afc talk page and at the Teahouse, and other more experienced editors (usually Huon), would take action and I would see what was done and know what to do next time in that situation. —Anne Delong (talk) 23:14, 19 October 2013 (UTC)

The mentorship idea is just a special "private tutoring" case of Buffbills7701's school idea (see "his" section above). I'm not sure if the original idea of a training program is Buffbill7701's or someone else's, it's been floating around for weeks if not months. davidwr/(talk)/(contribs) 01:59, 20 October 2013 (UTC)

This is entirely reasonable. MER-C 04:35, 21 October 2013 (UTC)

I like it. 78.26 (I'm no IP, talk to me!) 11:54, 24 October 2013 (UTC)

Support This "many routes" idea sounds like the best suggestion yet. (I assume that, aside from the sixth or "discussion" option, the right could be awarded by any administrator.) --MelanieN (talk) 14:43, 24 October 2013 (UTC)

• Complete and total support - This is the greatest idea I've seen. While there should be some kind of sieve to limit inexperienced or even malevolent editors from reviewing articles and destroying things, you cannot ignore the major backlog of AfC articles. Writing up a rigid list of requirements that admins must dig through edits to find is laborious for all parties involved and will make the backlog worse. The new "right" should be more to keep bad reviewers out than let good reviewers in. öBrambleberry of RiverClan 15:11, 31 October 2013 (UTC)

Implementation detail suggestion by Davidwr - preventing moves

 Withdrawn per "law of unintended consequences" as pointed out below by stefan2 at 13:23, 19 October 2013 (UTC). How did I miss that possibility? davidwr/(talk)/(contribs) 15:44, 19 October 2013 (UTC)

Should there be a consensus to prevent non-approved editors from accepting articles, one way to do this is to bot-move-protect the WT:AFC page shortly after creation, then allow the AFC Helper Script to trigger a bot to do any required moves. This would not require any changes to MediaWiki software. davidwr/(talk)/(contribs) 01:21, 19 October 2013 (UTC)

I'm not sure I understand this suggestion. Isn't there more than one way to skin a cat, as ShinmaWa's question-section above points out? Use of the helper-script is optional, and I guess I don't understand why bot-move-protecting the WT:AFC page will add security. Is there really no other way to get an article created (or take an existing stub and get it renamed) without going through WT:AFC at all? 74.192.84.101 (talk) 06:15, 19 October 2013 (UTC)

• Oy vey... All of these unproductive subheadings and then multitudes of comments about stuff in other subheadings... I can't follow the thread of any of these discussions and this page has gotten way TL;DR overnight. Unless someone who has been following can create a convenience break with an overview summary of all the ideas in one section or get rid of all of the subsections above or re-arrange comments so that comments are in the proper section headings (very bad wiki-etiquette, please don't), I'm afraid I can't contribute to this discussion at this time. I just don't have two days to try and piece mail all of the badly fragmented discussions back together. Technical 13 (talk) 12:25, 19 October 2013 (UTC)
• Oppose If moves are made more difficult, then we are likely to get more copy & paste moves which violate the attribution requirement. --Stefan2 (talk) 13:23, 19 October 2013 (UTC)

I disagree with encumbrances necessitated by the weakest link or worst-case scenario. We ought instead to fortify an imaginable breech with effective countermeasures; which do exist.—John Cline (talk) 03:20, 20 October 2013 (UTC)

Comment by James500

I was not aware of the existence of the previous RfC and I suspect that many others were not aware either. Consensus can change, so there is no reason why we cannot now discuss whether any new permissions are needed at all. James500 (talk) 20:03, 19 October 2013 (UTC)

Well alternatives could be a whitelist or a blacklist of those who can or those who can't. Other ideas were a series of awards to indicate progress or achievement. And there should be hat for the hat collectors. We already have barnstars and a listing of project participants. Perhaps someone can vet the newly added names to see how they are going. Graeme Bartlett (talk) 20:44, 19 October 2013 (UTC)

That's been dicussed further up. The problem is that currently, no reviewers are obliged to enter their names on that lit, hence we do nots always even know who is reviewing until problems come to light and are brought to the AfC talk page. Most issues are handled locally on the reviewer's talk page: 'Why did you decline my submission?' which begs the question: Why was the creator not provided with more detailed information?.

Hat collecting is an unavoidable but necessary evil. We get plenty of them at WP:PERM but we are fairly good at filtering them out. Kudpung กุดผึ้ง (talk) 03:23, 20 October 2013 (UTC)

The 'why did you decline my submission' enquiries occur even if you do explain the reasons when declining. Particularly troublesome are autobiography and WP:COI as a decline (even on material previously declined by another reviewer) often gets a flood of WP:OTHERSTUFFEXISTS and "but I worked so hard" pleas as to why the author really deserves to have their own article. This will continue even if you create roadblocks to entry for new reviewers. K7L (talk) 21:56, 20 October 2013 (UTC)

I don't like any of the proposals that have been made, but I think that a blacklist of those who have demonstrated that they are incompetent, compiled by human beings, would be the least worst option. James500 (talk) 07:06, 24 October 2013 (UTC)

Suggestion by TheOriginalSoni

Of all the solutions, I found Ross Hill's solutioon to be the most practical and useful, and hence I propose the following steps for selecting the AFC reviewers, based on a few adjustments I think should be made on it

• A selected few active and/or trusted reviewers will be grandfathered in. The exact details of how it will be done will be decided later.
• Anyone who wishes to become an AFC reviewer would be submitting their reviews of at least 10 articles currently at AFC. This would be done at a special requests page for
  • The review must be clear on why it is rejected, and any other such comments.
  • There must be at least 3 declines and 3 approvals among those submitted reviews
  • The reviews should be among pending submissions at the time of submission
  • Once any particular review is submitted, there should be no changes to it.
• These reviews are all open for comment from any current AFC Reviewers, who may choose to "Endorse" or "Disagree" with a particular review.
• Any article among the list which gets rejected or accepted externally would auto-count as an endorse or disagree by itself.
• After a period of time/ after all the reviews have been looked into, a designated person (the qualifications of which will be decided in future discussions) would close the request as pass or fail. In general, 8 or more correct reviews would count as pass, and 6 or less correct ones would be a fail.
• ^{[Additional proposal under discussion]} Any sufficiently trust candidates who have demonstated enough competence might not be required to go through this process, but handed over the tools directly on request.

This is the general schematic of how I think it should proceed. Every specific point in this suggestion is open for discussion, and would be altered as per consensus and common sense. TheOriginalSoni (talk) 00:42, 21 October 2013 (UTC)

Support Because reviewing skill is what matters, so that should be what we test. Ross Hill (talk) 00:59, 21 Oct 2013 (UTC) 00:59, 21 October 2013 (UTC)

Support as ONE way Oppose as THE ONLY way. Anyone who comes in having already demonstrated competence regarding content guidelines/policies and who doesn't have anything negative should be given a pass on this. Basically, any editor who has an edit history that would make them a credible candidate at RfA (I didn't say he would pass, just that he wouldn't be WP:SNOW-closed or otherwise fail miserably) and who doesn't have any thing negative in their recent history should not be required to do more than ask for access to the tools. The same goes for editors who might fail miserably at RfA only for reasons not relevant to AFC work. davidwr/(talk)/(contribs) 01:57, 21 October 2013 (UTC)

• I failed to notice this point. I agree that WP:IAR should apply to obviously trusted candidates and they shouldn't have to go through the entire process. But at the same time, I wonder if there is any harm in having them go through this simple enough process. If others also agree to the additional proposal, I'd be willing to add it. TheOriginalSoni (talk) 09:33, 21 October 2013 (UTC)

Neutral Overall those points look good, however the no more than "7/10 one direction" rule is going to jump up and bite a lot of hopefuls. Hasteur (talk) 13:55, 21 October 2013 (UTC)

I'm assuming that the intent is for the candidate to have at least 3 approves and at least 3 declines among the 10 reviews, so that his competency on both approvals and declines can be evaluated. A person may be fine when evaluating an article that should be declined but he may routinely over-decline and mis-evaluate things that should be approved, or vice-versa. Too many errors in either direction is counter-productive. davidwr/(talk)/(contribs) 16:14, 21 October 2013 (UTC)

The problem I was trying to indicate is ast least in my experience, I do about 80 to 90% decline simply because it takes a lot of effort to get a submission up to the level that I would pledge my reputation to the submission by accepting it. Hasteur (talk) 23:43, 21 October 2013 (UTC)

• Even then, I am of the opinion that there should be some limit of this sort to check for both sides of whether the reviewer knows the policies. If your concern is indeed correct, maybe we could lower it to at least 2, but I wouldn't want to remove it. TheOriginalSoni (talk) 13:07, 22 October 2013 (UTC)

Sure, we need proof that the person can approve at least 3, and decline at least 3, and we want to have at least 10 example-decisions to look over. Rather than saying that they must have ten cases, and they must approve 3 of *those* ten, and decline at least three of *those* ten, instead make this the rule: There must be no more than 7 declines or approvals among those 10 reviews The ten selected example-reviews *must* include at least 3 approves, and at least 3 denials; note that the reviewer-candidate often may actually need to review more than ten actual cases, to achieve 3 of each type... but only ten selected cases (including at least 3 approves and at least 3 denials) will really "count" when determining whether the reviewer-candidate passes the examination. HTH. 74.192.84.101 (talk) 21:21, 4 November 2013 (UTC)

• @74 Does the current wording of the proposal make more sense, or should there be further rewording on it? TheOriginalSoni (talk) 07:45, 6 November 2013 (UTC)

Suggestion by Pol430

I worry that asking potential reviewers to conduct trial reviews leaves too much room for 'instructor creep' and who will assess their performance? One person or more than one? The idea strikes me as fertile ground for creating a 'priesthood of gatekeepers' or turning the process into pseudo-RfA. I think we need to keep the process as simple as possible with a fairly black and white set of metrics to work to. Also, whatever form this permission takes, it should be transparently requestable via a noticeboard in the same way as other permissions. In my involvement at AfC, I have found that reviewers need to be able to demonstrate the following essential qualities:

1. Must be able to judge what constitutes vandalism, attack pages, and wholly negative unsourced BLPs
2. Must be able to identify copyright violations
3. Must be able to recognise WP:ARTSPAM and blatant hoaxes
4. Must demonstrate a sound understanding of notability, verifiability/reliable sourcing, and the BLP policy
5. Must be able to communicate with patience and clarity with new editors

I believe that these qualities would be best evidenced against the following criteria:

1. Must have carried out at least 50 good vandalism reverts -- a common threshold for granting of rollback (includes the speedy deletion of pages as blatant vandalism).
2. Must have correctly identified more than 5 attack pages or wholly negative BLPs, by whatever means.
3. Must have correctly cleaned up 20 articles with copyright concerns or correctly nominated 15 pages for speedy deletion as blatant copyright violations.
4. Must have copy edited/cleaned up at least 20 articles to make them NPOV compliant or correctly nominated 15 pages for speedy deletion as blatant spam/advertising.
5. Must have participated in at least 20 AfD discussions and !voted/commented with correct policy-based observations that demonstrate knowledge of notability, verifiability and reliable sourcing.
6. Must have demonstrated a sound knowledge of BLP Policy issues, by whatever means. For example, working at the BLP noticeboard.
7. Must have demonstrated an ability to help and work patiently with newer editors. For example, tea house host, adoption, help boards, user talk page assistance.

These minimum criteria could be assessed by any administrator patrolling the noticeboard, but should be rigidly applied. In the case of the right/permission being abused, any administrator may remove the right as a discretionary sanction in the same manner as other rights. Pol430 talk to me 18:00, 21 October 2013 (UTC)

I've been an editor and AFC-participant for many years and I don't think I meet all seven of the items on the bottom list, and I know that I have weaknesses in notability in certain subject areas and an inability to communicate with patience and clarity with certain editors. I'm also not as good at detecting advertisements disguised as articles as I would like, but I am getting better at that with experience. davidwr/(talk)/(contribs) 19:51, 21 October 2013 (UTC)

I'm pretty sure that you have made 50 good reverts in your Wiki career and most of the CSD criteria would be easily evidenced by someone who spent a few months patrolling new pages. Equally, participation in 20 AfD discussions is not hard for most long-serving Wikipedians to evidence. Knowledge of BLP policy can be demonstrated by various means and I think your interaction with users on your talk page demonstrates point 7 just fine. I think notability is an area that AfC can sometimes get a little hung up on. In terms of notability, AfC's job is to keep out articles about obviously non-notable subjects; this includes cases where a very solid policy-based argument for not including a subject can be made. Where notability is borderline, then articles need to have the opportunity to receive community discussion about their inclusion in Wikipedia, this means accepting a submission without prejudice to it being nominated at AfD. In cases where notability is difficult to establish because of the specialised nature of the subject area then help may be forthcoming from a relevant Wikiproject. If not, we still have an obligation to AGF and accept a submission without prejudice to an AfD nomination -- that is where the responsibility for ruling definitively on a subject's notability lays. Pol430 talk to me 16:40, 22 October 2013 (UTC)

Huh? First you said "keep it simple" and then you came up with the most complicated possible process. You laid out seven specific numerical criteria which you think should be "rigidly applied". Who in the world (either applicant or administrator) is going to go through histories counting how often someone has cleaned up copyvio or identified attack pages? This process would be unworkable, and furthermore it is not based on any evidence that these things would matter. I agree with "keep it simple", namely, let administrators review the person's contributions and decide if they seem competent enough for the task. Period. --MelanieN (talk) 21:39, 23 October 2013 (UTC)

Good luck with trying to search my 30,000+ contributions to check that I pass all seven "must have" criteria. Roger (Dodger67) (talk) 09:58, 29 October 2013 (UTC)

I recognise that cross-checking those seven criteria would be laborious, but I thought the purpose of the exercise was to ensure high standards rather than dish out a new hat as quickly and widely as possible. I have struck out the criteria that were so evidently wide of the mark. Thanks for your feedback. Bellerophon _{talk to me} 00:10, 31 October 2013 (UTC) (formerly Pol430)

Comment by Brambleberry of RiverClan

The list of articles waiting to be reviewed is backlogged enough, so while I agree that we do need something to make sure reviewers are qualified, those standards should not be so high that only a select few can access them. I think that the criteria should be rather vague, leaving it up to a case-by-case basis. There can be a few strict ones, like a certain amount of article space edits, but things such as "must have rollback and/or reviewer rights" seem a bit too constricting and would be thoroughly unconstructive to the main purpose: reviewing articles to add to Wikipedia. öBrambleberry of RiverClan 23:48, 21 October 2013 (UTC)

I agree with this. The process should be simple, not stringent; we are simply trying to stop the current situation where unsuitable and/or inexperienced editors are trying (mostly in good faith) to review at AfC without sufficient knowledge of Wikipedia policies. IMO the AfC Reviewer right should be awardable by any administrator who believes the user has sufficient relevant experience to know an acceptable Wikipedia article when they see one. We trust administrators to make far more difficult/controversial decisions than this, and I don't really think Wikipedia will suffer any harm from letting them use their judgment in awarding this right. The qualifications for Reviewer, as listed at the top of this discussion by Kudpung, would serve equally well as qualifications for AfC Reviewer, but the one should not be a prerequisite for the other. As for Rollbacker, that right is both trivial and annoying; personally after having it (and cussing at it) for six months I asked that it be removed. Presence or absence of a Rollbacker right does not in any way reflect the user's ability to review submitted articles. --MelanieN (talk) 21:31, 23 October 2013 (UTC)

Part of the reason for the backlog is that many experienced users will not participate. One reason is the many useless, complicated ,and counterproductive procedures in the AfC process, for which see my user talk special archive. But the main reason is that unless most reviewers are moderately competent, what a good reviewer can contribute will be wasted. there's no point in contributing to processes which work poorly and on which one can not make an impact. Otherwise there is a much higher priority for anyone who knows what to do--which is checking their work, and trying to teach those who most need it. Ten good people without interference from the unqualified can do the process better than ten good ones trying also to cope with fifty unqualified.

If we cannot get high standards, we will need to see this only as a first screen, and the accepted articles are going to have to go into NPP so they will be checked a second time. As for the wrongly rejected, they will mostly continue to be lost to us. DGG ( talk ) 04:58, 25 October 2013 (UTC)

If the commonly used tools do not maintain a log page, they could be modified to do so. Gigs (talk) 18:43, 28 October 2013 (UTC)

DGG, a very accurate summary, and so much of it applies equally to NPP. Allowing them through to NPP would be counter productive, the NPP system has the same kind of flaws as AfC and there is no guarantee that the patrollers, who need no qualifications at all, will pass or tag such an article correctly. Most worrying of course, are the 'lost' incorrectly rejected articles, while a significant concern is whether articles are correctly checked for spam or copyvio etc. (BTW: I have taken the liberty of correcting the link to your talk page archive - that thread is very important and although long, I would recommend the participants here to read it). Kudpung กุดผึ้ง (talk) 07:09, 29 October 2013 (UTC)

A quick note regarding wrongfully rejected articles – that is also an editor retention issue ... if newcomers are being encouraged to send articles to AfC under the guise that "an experienced reviewer will double-check everything before sending it to mainspace" (which is usually what they are told at places like the Teahouse, and I at times am the one telling them that), then that is a problem. I think DGG's comment regarding sending pages to NPP may have a valid point, however agreeing with Kudpung, NPP draws inexperienced reviewers as well, but at least if you have two chances to catch a problem, that is better than only one. Nevertheless, the more I read about this and think about this, the more I am convinced that the new AFC academy is a good idea if people actually use it and have a desire to do things correctly, and frankly, perhaps a user right is in order. I would think anyone trusted with reviewer or autopatrolled would have necessary qualifications to review new articles, but I don't know for sure. I know user rights mean more bureaucracy, and a manufactured debate over the haves and have nots, but at the end of the day, bad reviewing of AfC and NPP has ramifications on copyright, editor retention, and, perhaps most importantly, missing content that can fit into the breadth of the world's knowledge; that is what we need to protect in these discussions. We can discuss whether someone should have 500 edits or 750 all day long, but that is not what is important. We need people who simply know what they are doing, and if they are doing things wrong, we can firmly, yet gently suggest they utilize training of some kind, and if they refuse that, they simply must be told to stop, as they hinder progress. I think the best way to maintain our NPP and AFC processes would be if experienced editors – article writers, content gnomes, admin, etc. – would all simply commit to reviewing x articles per week, and keep up with it. That would prevent massive backlogs, improve the quality of the reviews, and reduce potential ensuing burnout from one person trying to simply bust the backlog on their own. Go Phightins! 10:46, 29 October 2013 (UTC)

Another very appropriate comment. However, have you tried herding cats? It works, but you have to give them something. Kudpung กุดผึ้ง (talk) 11:04, 29 October 2013 (UTC)

@Go Phightins!: totally agreed. -- phoebe / (talk to me) 19:29, 30 October 2013 (UTC)

Comment from Phoebe

I don't have an opinion on the technical details of a right, how it is implemented, what the threshold is, etc. But I am worried, in general, about the impact of AfC on new users. The difference between a constructive, welcoming review -- even if the article isn't really up to par, a constructive review can still be made -- and a snarky or brusque one (made for whatever reason, including working fast because of the backlog) is huge for a new contributor. If a right can help with standards-setting among AfC reviewers on interacting with new users -- and maybe even make AfC a more attractive place to participate for experienced editors -- then I'm all for it.

I am well aware of our backlogs and the huge amounts of spam etc. But AfC is also a touchpoint for hundreds of new contributors who if they make it to AfC in the first place are generally also well-meaning and interested enough to perhaps be converted into active editors. Currently, AfC is sort of a Wikipedia backwater, and I feel like it should be front-and-center as a place for us to triage and work. I think a right if done well *might* help with this -- so to the extent it does, I'd support it. (If, however, it turns into simply 'one more collectible thing' as someone else pointed out, or somehow limits participation in AfC by existing helpful reviewers, then I wouldn't). -- phoebe / (talk to me) 19:27, 30 October 2013 (UTC)

I certainly agree that it should be a 'front-and-center' operation and akin to the importance of NPP which was granted a complex set of tools by the Foundation. However, alone the 60,000 abandoned G13 drafts, of which I have physically deleted several hundred, demonstrate that what comes through AfC includes a vast amount of totally unacceptable junk, often far worse that what comes through NPP, and the fact that the creators have gone through the Article Wizard or AfC does not prove at all, unfortunately, that they are all good faith submissions. At AfC there is a cohesive and supportive dedicated team driving things forward; NPP has nothing of the kind bar its instruction page, has a talk page that sees a message once in a blue moon, needs no qualifications, and suffers from the same ailments as AfC. Kudpung กุดผึ้ง (talk) 04:42, 31 October 2013 (UTC)

NPP has one advantage over AFC: By definition, every page they looked at was created by an autoconfirmed or confirmed editor. davidwr/(talk)/(contribs) 15:21, 31 October 2013 (UTC)

by G13 drafts you mean random userpage drafts, right? 1) If so, I don't know why you or anyone else would delete these; userpages are meant for drafts, and perhaps except in extreme libel situations or similar are doing no one any harm even if they're not destined to be good articles. 2) Not sure how this relates to the good-faith-edness of AfC. When I teach people how to edit, I tell them to start in their sandboxes. By your measure, if their userpage drafts aren't up to speed they're not contributing in good faith? That makes no sense; these are two different measures. Good faith is largely unrelated to whether the article is complete, referenced, notable, etc. And I've looked at enough AfC submissions myself to be pretty sure they're not all "Johnny sucks" or similar. -- phoebe / (talk to me) 15:43, 31 October 2013 (UTC)

SeeWP:CSD#G13, a relatively recent speedy deletion criterion which specifies that AfC submissions which have not been edited at all (not even a keystroke) in more than 6 months may be deleted. DES ^(talk) 16:00, 31 October 2013 (UTC)

Yes, there are many good (or eventually good) submissions to Afc. As well as beginning editors, it's also widely used by COI editors who want to make sure that their articles won't be deleted as advertising. Afc reviewers help tone these down. To date there have been over 34,000 successful accepted submissions, and most of them left Afc in far better condition than when they arrived. Here are the ones accepted this month: CatScan report Also, articles eligible for deletion under the G13 criteria aren't always deleted; there are a number of editors who are checking through them and picking out ones to improve. —Anne Delong (talk) 08:20, 1 November 2013 (UTC)

Thanks for the pointer to the CatScan tool Anne. That's very useful. Steven Walling (WMF) • talk 05:52, 7 November 2013 (UTC)

Research needed?

Hey all. I don't have an opinion about the RFC in question, at least in an official WMF capacity. However, my team is just barely beginning to explore potential improvements to article creation. As a part of this, myself and our research scientist are working on measuring the current state of article creations and creators each month. That includes the volume at AfC, which though unique to English Wikipedia, is obviously an important route for new authors here. If you can help us think of strategies for accurately measuring the number of submissions, as well as decline/accept rates, that would be most welcome. Our notes are at Research:Wikipedia article creation. Many thanks, Steven Walling (WMF) • talk 05:57, 7 November 2013 (UTC)

Technicalities

Back on track, or partly - because there are mixed opinions whether an MedWiki-independent solution could be achieved, there is something for our resident programmers to look at: here. Other ideas may be coming soon, but I still feel that a set of criteria comes first, then to see how they can best bee implemented. Kudpung กุดผึ้ง (talk) 03:05, 5 November 2013 (UTC)