Wikipedia talk:Pending changes/Trial

Talk pages

As a check against manipulation by outside groups, and to preserve the free flow of ideas, I think it is important to say explicitly that Talk: pages should not now nor ever be subject to anything but passive patrolling, even for BLPs. Because Talk: pages are not now subject to full or partial protection, I think there should be broad consensus for this. Wnt (talk) 17:10, 19 January 2010 (UTC)

The existing policy on this isn't changing, but it makes sense to be explicit. --Elvey (talk) 23:17, 8 February 2010 (UTC)

The trial is limited to articles, and I don't think that the extension FlaggedRevs allows to flag talk pages anyway. Cenarium (talk) 17:02, 27 March 2010 (UTC)

I believe that is correct. On the testing site flagged protection can only be enabled in namespace 0 (the article namespace), and I assume that's how it will work here as well. Reach Out to the Truth 19:41, 27 March 2010 (UTC)

Range of articles we need to test this on

Idealy we need to test this on:

• Articles under full protection
• Articles under semi protection
• Articles under neither but that we suspect will have BLP problems (keep an eye on the news to generate this list).
• Control groups.

Genisock2 (talk) 17:19, 8 May 2010 (UTC)

That looks like a reasonable list Genisock2. Could you work that into the text of Wikipedia:Flagged protection and patrolled revisions/Trial? -- RobLa (talk) 19:02, 9 May 2010 (UTC)

Keep in mind that as established in the consensus found for this trial we should respect the protection policy in choosing where to use flagged protection. Preemptive protection for example is a no go (except in truly exceptional instances). Cenarium (talk) 00:34, 14 May 2010 (UTC)

Simplified protection levels for the trial

After a few private conversations and trying to get my head around the proposed levels and policies, it looks to me like there's some pretty complicated extra user access levels that are being proposed, which seem a bit much given the level of experience this community has with the feature. So, I've proposed a simpler set of access levels for the trial, with the idea that none of this is permanent, and we'll be able to add in a new layer should a consensus emerge that a new access level is necessary to go forward. -- RobLa (talk) 20:15, 10 May 2010 (UTC)

Categories for edits needing review

How are we planning on listing edits needing review? Can we list them by Wikiproject? I would be primarily interested in reviewing medical edits and less in reviewing edits in general. Also how will it be decided what pages get flagged? Will this be lead to the discretion of the Wikiprojects within certain guidelines?--Doc James (talk · contribs · email) 04:44, 29 May 2010 (UTC)

Success criteria?

We have 4 days to go and don't yet know what the criteria for measuring success of the trial are? Rich Farmbrough, 18:20, 11 June 2010 (UTC).

Hopefully we can measure a bunch of stuff such as:

• Time between edit and review
• Volume of pending changes
• Change in vandalism frequency per and post implementation on pages not previously protected
• Number of suggestions on pages which were previously semi protected

We have this data from before "42% of vandalism is repaired within one viewing and 11% is still present after 100 viewings." Doc James (talk · contribs · email) 18:48, 11 June 2010 (UTC)

Closure

How should the trial end? It had been decided that at the end of the (two-month) trial, the community should decide on whether to continue the implementation or not, with consensus needed to continue it. So a discussion should happen at the end or before the end of the trial, but it hasn't been decided what to do of the implementation pending the conclusion of the discussions, i.e. should it be entirely deactivated, put on hold, or still used.

Comments

I guess I'll start by brainstorming some possible outcomes, and then ask for the right metrics on those outcomes:

Postitive outcomes

• Previously semi-protected articles receive a substantial number of accepted edits from anonymous and new users
• A substantial number of malign/poor edits from autoconfirmed non-reviewers are reverted on review of previously semi-protected articles, thus never becoming visible
• Good edits to pages under 'pending changes' are accepted quickly

Negative outcomes

• Previously semi-protected articles receive fewer edits from autoconfirmed, non-reviewers.
• Good edits to pages under 'pending changes' are accepted slowly
• Editors who make their first edits on pages under 'pending changes' are less inclined to stick around

The correct thresholds are hard to define in advance, and I have no idea if how hard the metrics are to come up with (I'll ask around). The point is to come up with metrics that illustrate the most important benefits and risks of this new feature. It's virtually certain we're not going to be able to pull together all of the data we want, so we need to quickly narrow down the subset of most important information that is realistic to gather.

As for figuring out the consensus, we could conduct a !vote with two separate questions:

• Should 'pending changes', in its current form be kept or removed from English Wikipedia (keep/remove)?
• Should work on future versions of 'pending changes' continue, or stop for now (continue/stop)?

Possible outcomes:

• keep/continue - this means the current configuration will remain on, and work to refine the implementation and configuration will continue
• keep/stop - this means we keep the current implementation and configuration, and largely stop futzing with it for a while
• remove/continue - the means we should take the current iteration down, and work to refine the implementation and configuration for a future trial
• remove/stop - this means we remove the current implementation, and stop work on this for now

Does this sound like the right framework for having an informed discussion about retaining/removing 'pending changes' at the end of the trial? -- RobLa (talk) 20:48, 6 June 2010 (UTC)

Yes, it looks good. But, assuming the trial ends circa August 15, we'll need to have some discussion before the vote, say 1 week, and the vote itself should at least stay opened for 2 weeks, so it'll take at least 3 weeks to reach a conclusion. Until we reach the conclusion, what should we do of the implementation ? Should it be frozen (no new articles under PC), or entirely deactivated ? Cenarium (talk) 02:39, 13 June 2010 (UTC)

The trial IMO should be left running well we decide whether or not to implement it long term. Turning things on and off gets confusing.Doc James (talk · contribs · email) 22:33, 13 June 2010 (UTC)

Good edits to pages under 'pending changes' are accepted quickly

I'm not sure that the trial actually tests this, as there is going to be an absolute limit to the pages that can be protected in this fashion which is probably substantially lower than the number of pages likely to be so protected were the scheme to be approved. Espresso Addict (talk) 01:34, 14 June 2010 (UTC)

Agreed; the ratio of reviewers to pages is something like 100 to 1 right now. Until we have a critical mass of protected pages, there's no way to tell how our response time will be. — Carl (CBM · talk) 14:31, 16 June 2010 (UTC)

Bots

Bots could be helpful for the trial. I'm thinking of two tasks, if they are possible:

1. reporting edits by autoconfirmed users who are not reviewers to level 1-pending changes-protected pages with pending edits
2. reporting pending edits not accepted for more than n minutes, where n is fixed depending on the average backlog
3. holding as many statistics as possible, which can't be hold by the software itself
4. all autoconfirmed users editing pages protected with pending changes (not only when they have pending edits) in list format.

Cenarium (talk) 08:38, 14 June 2010 (UTC)

How are you thinking of "reporting" these things, on an IRC channel or in some kind of list/category on a Wikipedia page? - EdoDodo ^talk 19:00, 14 June 2010 (UTC)

On a wikipedia talk page or a noticeboard, the bot would report the edits under a fixed section, in a way similar to WP:AIV. Cenarium (talk) 21:26, 14 June 2010 (UTC)

In my opinion, that's a very good idea! Salvio ( ^{Let's talk 'bout it!}) 22:28, 14 June 2010 (UTC)

Well, they could also be reported in a IRC channel such as the one for pending changes. Cenarium (talk) 03:48, 17 June 2010 (UTC)

We'd also need to know the precise number of articles under pending changes. Cenarium (talk) 23:19, 14 June 2010 (UTC)

So we can control the number of articles under PC, the bot would just update a template with the current number. Cenarium (talk) 17:41, 15 June 2010 (UTC)

I could probably manage (3) if you were to give me more information on what you need. It seems to me to also be the most pressing. I'd leave (1) and (2) for others with more time/experience. - Jarry1250 ^{[Humorous? Discuss.]} 18:16, 15 June 2010 (UTC)

How is this data available? Can we get it from the API? — Carl (CBM · talk) 19:25, 15 June 2010 (UTC)

The extension is described at mw:Extension:FlaggedRevs, it mentions API. Cenarium (talk) 13:59, 16 June 2010 (UTC)

I spent a while looking at the API yesterday. It has some output, but not everything that might be desired.

I'm not really sure what bot tasks you are looking for here. There is already a collection of special pages that seem to list everything, and some of the things you suggest sound like they would be better as special pages (with dynamic updating).

What bot tasks, exactly, are you thinking of? — Carl (CBM · talk) 14:27, 16 June 2010 (UTC)

1. It's important to know if there are edits by autoconfirmed users who are not reviewers to level 1-pending changes-protected pages with pending edits, so I wonder if a bot could detect them and report them. I'm not familiar with the API and how bot works; it seems the API allows access to the pages listed at Special:Oldreviewedpages, see mw:Extension:FlaggedRevs#list_.3D_oldreviewedpages, so it may be able to detect each new edit to one of those pages by non-reviewer autoconfirmed users ?
2. It's also important to know if there is a backlog of pending changes, could a bot warn if there is a pending edit which has not been reviewed for more than x minutes (say 5 for now), also using Special:Oldreviewedpages (by checking the timestamp for the latest accepted revisions) ?
3. Also requested: a bot which updates a template with the number of pages under pending changes, which can be retrieved from Special:StablePages. Cenarium (talk) 03:48, 17 June 2010 (UTC)

All of these would be better if they were implemented directly in mediawiki, rather than done as a bot. For #1, I don't understand the difference between what you are saying and Special:Oldreviewedpages.

For #2, the problem is that if the bot posts a message "there are old edits" and someone reviews them, the bot's message is instantly outdated. Even if the bot ran once per minute, many people would see the message. So it would be better to just have a special page that shows the pages in order of "oldest unreviewed revision".

For #3, it is just a matter of adding a new magic word to the system. — Carl (CBM · talk) 11:12, 17 June 2010 (UTC)

There is a bug filed (feature request) for #3 at bugzilla:23903. — Carl (CBM · talk) 11:39, 17 June 2010 (UTC)

Special:Oldreviewedpages lists pages with pending edits. What we need to gather are edits made by autoconfirmed users who are not reviewers to pages with pending edits. Because on semi-protected pages, autoconfirmed users would be able to edit without restriction, but if they're editing a PC-protected page while the latest edit was by an unregistered or new user and not yet accepted, their edit will be delayed. So we need to be aware of such events.

I suppose the bot could check if there are several old edits, for example more than 5 edits older than 5 minutes. For now it's not going to be a problem, I rarely see a page listed at Special:Oldreviewedpages, but when we'll have more pages under PC, backlogs may happen occasionally, so we'll need to attract the attention of reviewers. Cenarium (talk) 23:17, 17 June 2010 (UTC)

Testing pages

We'll have pending changes available in WP namespace so we can make testing; we can use subpages of Wikipedia:Pending changes/Testing, e.g. Wikipedia:Pending changes/Testing/1, Wikipedia:Pending changes/Testing/2, etc. There should be some content on them, we could copy a few articles (uncontroversial ones, like ancient history). Cenarium (talk) 23:30, 14 June 2010 (UTC)

A stupid question, I realize, but I'll ask anyway: wouldn't there be a problem with attribution? Salvio ( ^{Let's talk 'bout it!}) 13:23, 15 June 2010 (UTC)

Not if you say from where it comes. At http://flaggedrevs.labs.wikimedia.org/wiki/Main_Page pages are articles imported from enwiki. Cenarium (talk) 15:24, 15 June 2010 (UTC)

I had anticipated it was a stupid question... Thanks for your answer, however. ;) Salvio ( ^{Let's talk 'bout it!}) 17:15, 15 June 2010 (UTC)

Entering in a reason

It queries you for a reason whether you accept or decline an edit. Where are those summaries stored and how does one access them? Mkdw^talk 06:53, 16 June 2010 (UTC)

I found them on the page logs but in order to access it you have to view history -> logs -> review logs. Quite the numerous steps for people that will surely want it to be readily available. Perhaps review summary could show up on a mouse over, or have the summary written in beside the accepted tag. Mkdw^talk 06:58, 16 June 2010 (UTC)

Bug

Reverting changes with Twinkle doesn't work. --MrStalker ^(talk) 16:40, 17 June 2010 (UTC)

Leaving Pending Changes on during the discussion and !voting period

We don't yet have an exact timeline for when a !vote will occur and when we'll have all of the statistics gathered for an informed decision on whether to keep this feature. The one date that is fixed is August 15, when the two-month trial officially ends. I'd like to suggest that during that brief period while we figure out what the permanent plan is, we leave the feature running. It'll be less operational headache and give us a place to still view the feature in action while a vote is taking place. -- RobLa (talk) 22:10, 18 June 2010 (UTC)