Wikipedia talk:New pages patrol/Analysis and proposal

Thank you

Thank you to the folks who put this together. I quickly scanned the analysis, and while I don't agree with a few of its assumptions, it does shine a light on the need to find a balance between thoroughness and speed. I will probably have more comments or questions after I read it more carefully.- MrX 23:43, 31 May 2017 (UTC)

Graph question

Thanks so much for this thorough consideration. I look forward to thinking through and discussing together. If I may ask one informational point to begin (forgive me if this should be obvious). As to the key in the graph, saying blue indicates "users who are still new (not autoconfirmed) today". Does blue then indicate only from pages from editors who were still not autoconfirmed when you generated this graph on May 25, or all pages from editors not autoconfirmed at the time of their entry's creation? I.e. if four months ago, a not-yet-autoconfirmed editor made a page; it's still in the backlog; but that person subsequently made ten edits to WP, becoming autoconfirmed: is their backlogged entry marked green or blue in the graph? I ask because this would, of course, change how much the graph tells us about what percentage of the backlog would be affected by requiring autoconfirm at the time the editor creates the page.

Thanks again for the sustained attention to this important challenge. Innisfree987 (talk) 23:46, 31 May 2017 (UTC)

Yes, thanks for asking. We're working on a second version that looks at the time of creation; we should have it pretty soon, and we'll add it to the report. DannyH (WMF) (talk) 01:04, 1 June 2017 (UTC)

Great, thanks! Innisfree987 (talk) 01:19, 1 June 2017 (UTC)

@Innisfree987: To answer your question, the blue part of the graph indicates pages created by editors who were still not autoconfirmed when we generated the graph on May 25. Unfortunately, it's quite difficult to generate numbers based on the editor's status at the time of article creation, but we are still working on trying to get numbers for that. Kaldari (talk) 20:29, 1 June 2017 (UTC)

Yes, "today" in that legend refers to current times (more precisely, 0:38 UTC on May 25), so your first interpretation is correct. To explain why the graph uses that definition, and also to clear up some misunderstandings elsewhere:

This graph represents a snapshot of the existing review backlog as it appears to new page patrollers at Special:NewPagesFeed. In particular, it uses the same definition of "new editors". I.e. if you change settings there to only show unreviewed articles by "new editors", you should get a list that exactly corresponds to the blue part of the graph.

As to why this existing backlog display interface at Special:NewPagesFeed uses a definition of "new editors" based on their autoconfirmed status today rather than at the time of article creation: This is a deliberate choice in the software - it contains code written specifically for the purpose of updating an article creator's status in Special:NewPagesFeed once they reach autoconfirmed status after creating the article. I wasn't around in 2012 when this decision was made, but I can imagine various arguments for and against it. If people want to make a case that the other alternative (status at article creation) would actually be more useful for patrollers, I imagine that a Phabricator request to change it would receive fair consideration.

Back to the chart: As Danny says, we had already been working on a version based on the alternative definition, because that's more pertinent to the specific question of how much the backlog would be reduced if non-autoconfirmed users were prevented from creating articles. (I have by now written and tested a query for this. But based on some speed tests on a smaller amount of data, it may take several days to run fully, and could make our DBAs unhappy, so I'm still working on optimizing it further. In any case, we should have a result next week.)

However, we don't expect the result of this refined query to differ too much regarding the overall takeaway that new users - by either definition - are only responsible for a small part of the backlog. This is based, for example, on a manual check I did last week, examining 65 articles from the backlog (selected arbitrarily from those created on February 15 and March 15), of which only 3 were by new users who had subsequently become autoconfirmed. That's why we felt comfortable using the first version of the chart (the one consistent with Special:NewPagesFeed) in this report already.

PS: This is all based on public data, and the PAWS notebook used to create the graph is linked in the report, so anyone can reproduce and modify it - I'm happy to help. I think the graph's legend was fairly unambiguous already, but I also just updated the file description page with more detail and the link to the notebook. Regards, Tbayer (WMF) (talk) 01:45, 2 June 2017 (UTC)

Tbayer (WMF) and Kaldari, thank you both for the detailed information about the data available. All makes sense; in fact I'm pleasantly surprised it's possible, I wasn't sure if the necessary info was being logged (e.g. when I check an individual's user right log, it doesn't show when someone becomes autoconfirmed). It'd be great to see it if possible--I'd definitely be interested to know if we can confirm new users' creations are really a small part of the backlog, rather than these only appearing to be a small part of the backlog because the authors later made a few more edits and their contributions turned from blue to green in the graph. Thanks for the efforts to pin it down! Innisfree987 (talk) 04:13, 2 June 2017 (UTC)

Question on numbers

Tracked in Phabricator
Task T166269

First, let me say thank you for taking the time to write this and provide us with some numbers. From my first glance through it appears to me that the report is missing what is one of the more important numbers in the discussion: the number of pages created within the last 90 days that have been deleted, and then breaking that out by pages created by autoconfirmed and pages created by non-autoconfirmed. I'm still digesting the report and intend to read it again before making any substantial comments, but I do think that these numbers are critical to the conversation moving forward. TonyBallioni (talk) 23:53, 31 May 2017 (UTC)

That's a really good question. I'll see what we can find out, and I'll get back to you (or somebody will) soon. DannyH (WMF) (talk) 01:03, 1 June 2017 (UTC)

@TonyBallioni: We have an outstanding Phabricator task for getting numbers on that: T166269. Hopefully, we should have an answer by next week. Kaldari (talk) 20:32, 1 June 2017 (UTC)

I just want to say, in case someone makes the same assumption I did: I know it seems like these numbers should be easy to get. But the Vienna Hackathon was a real eye-opener for me on that score. I worked alongside numerous very skilled technical people who were trying to get this and other numbers, with unsatisfactory results. So thanks for your patience on this. JMatazzoni (WMF) (talk) 21:06, 1 June 2017 (UTC)

Rewrite the lede

Thanks for this analysis. Your data runs counter to what I assumed the problem has been. I still support WP:ACTRIAL but I also think most people shouldn't be editing Wikipedia, either. I don't think the best solution to "Time-Consuming Judgment Calls" is to let them age-off the list. I understand the logic presented but NPP is our sorting mechanism for problematic content. To my mind it would make more sense to have the "Time-Consuming Judgment Call" articles nominated by a bot (like WP:G13) for deletion after 60 days so that WP:DELSORT can assemble the subject-specific experts to make a determination. Orphaned, dead-end content (if not addressed by NPP) could remain out of sight for years until finally addressed. To that end, I think it advisable to re-write the lede to not include your conclusion, as reading that up-front almost blew all of my buy-in and I was about to write another dismissive denouncement of all of you in San Francisco. I'm glad I read the piece to the end and your presentation makes some difference as to how we proceed, but I could barely stomach that conclusion. Please bury it if you're going to provide it. Chris Troutman (talk) 00:41, 1 June 2017 (UTC)

Well, I'm glad you kept reading, and that we're talking about it. You bring up some fundamental questions about not just NPP, but about how Wikipedia as a whole looks at new content: Do we assume that new content is bad until specifically approved, or that it's okay unless it gets challenged? If marginal content remains out of sight until someone finds it and improves it, does that harm the encyclopedia? It sounds like your view on that is different from mine, but it's good for us to see that, and have those conversations -- not just you and me, but everybody who's concerned about it.

That being said, I don't know much about DELSORT, or how that workflow works. How active are they, and what does their backlog look like? DannyH (WMF) (talk) 01:00, 1 June 2017 (UTC)

DELSORT is a system of pages that list AfD discussions about a given topic. For example, WP:VG/D lists video game-related ones. (These pages are listed at WP:DST.) Editors look at new AfDs, evaluate which topics it's related to, and add them to the relevant topic pages. The point is to get input on these AfDs from people looking at their topic's DELSORT page. (I wrote a popular script that simplifies the tagging process to a dropdown menu.) I'm not sure if we've analyzed any data on the effectiveness of DELSORT, but I assume it's pretty good at getting topic experts involved in AfDs. Enterprisey (talk!) 01:38, 1 June 2017 (UTC)

(edit conflict) @DannyH (WMF): DELSORT doesn't declare that they have a backlog but the page itself claims that AfD (which they support) has grown to mammoth proportions. AfD has also made PROD'ding easier which intones that there are too many cases where only a couple editors !vote. I don't think DELSORT would be too happy about an influx of nominees but AfD has editors that watch specific topics and could render judgement about subject-specific notability criterion. Although most fans of a particular topic (the likely responders at AfD) will probably tend to be inclusionist (resulting in a lot of keep consensuses) that's the crowd that would be most likely to improve said articles.

I think Wikipedia contains a lot of content that fails notability criteria often as the result of POV forks, promotional efforts, and fandom. It does harm the encyclopedia to assume the "Time-Consuming Judgment Call" articles are ok because that substandard content pops up in search engine results and is visible to the reader. I also see little value in retaining poorly-written articles, regardless of content, especially where they rob a serious editor of a future four award. The assumption that the aggregate will crowd-source a good encyclopedia ignores the motivations of editors, which is why I'm pessimistic about new content, generally speaking. Chris Troutman (talk) 01:44, 1 June 2017 (UTC)

I do tend to think AfD would be completely overrun if however-many-thousand unreviewed entries began falling over the waterfall into the AfD queue; but, I'd be very much in support of a DELSORT-like tool within NPP, to help connect experts to subject matter as the report suggests. Being able to filter the queue by category or WikiProject would be one approach. (Not every submission initially has categories or WikiProjects but those are very easy to identify and add, in contrast, as the report notes, to the onerous and legitimately difficult work of the full reviewing checklist.) I know I'd do a lot more reviewing--and actual improving of the relevant entries--if I could easily find "Time-Consuming Judgment Calls" related to my interests and expertise, instead of going through them at random, which does quickly become wearisome. Innisfree987 (talk) 02:06, 1 June 2017 (UTC)

I'd definitely oppose a fall-off date (see Pathways To Peace a G12 candidate that survived 7 years as an example of what we could be missing if we had an arbitrary cutoff). I also think that the broader philosophical question asked about the nature of new pages actually has an empirical side to it as I mentioned above: how many new pages are deleted with 90 days.

I suspect that number is not insignificant, and it doesn't even have to be a majority to be harmful to the encyclopedia if they fall off to not be touched until someone realizes that it was a cleverly designed attack page or a copyright violation years later. The deletion numbers are critical to this conversation because they shed light on what the community has judged not to be acceptable from the recently created pages. TonyBallioni (talk) 02:33, 1 June 2017 (UTC)

Happily, I think the copyright violations are now handled by another workflow -- CopyPatrol has a bot checking every new revision, and comparing it to a search database to find potential copyvio. Each case is checked by a human volunteer, and either deleted, fixed or marked as a false positive. Community Tech built a new interface for it last year, and there's a small but super effective team of people who clear all the cases within about a day. So the Pathways to Peace-style copyvio should already be caught by CopyPatrol. DannyH (WMF) (talk) 03:19, 1 June 2017 (UTC)

Question

I guess my question, statistically, is whether there is some "critical mass" of views by reviewers that means the the page should probably be "semi-auto-reviewed" because enough people have passed on it, that it is likely to not be nominated for deletion, likely to survive that discussion, and/or likely to survive long enough for someone to find it and improve it? If that is identifiable, it seems preferable to an arbitrary limit like 30 or 60 days. TimothyJosephWood 02:17, 1 June 2017 (UTC)

As a follow up, if this question isn't answerable, and I suspect it may not be, would it be possible to incorporate a "pass" into the curator, so that we might be able to have an option for a good-faith "I'm not 100% comfortable with the subject/subject area/language/ect, and I'm not reviewing, but I'm not taking it to XFD/PROD/CSD" and try to gauge at what point enough "passes" might suffice to remove it from the backlog? TimothyJosephWood 02:44, 1 June 2017 (UTC)

We tried pulling some stats on clickthroughs from Special:NewPages and Special:NewPagesFeed and looked for the 25 example pages from the report, but I didn't include it because we're not sure that it's catching everything. There may be some clicks from tools like Twinkle that aren't counted. Some pages do get lots of clicks -- Heath Hitler got 13 clicks from NewPagesFeed, go figure :) -- but we didn't dig into it very far.

I think that building in some kind of staged system could be a promising direction. We were surprised to find that 28% of articles had been edited (not just viewed) by reviewers without being marked as reviewed. That seems like wasted work -- there should be more signals built into the system than just "reviewed" or "not reviewed". A reviewer looking at a new page should at least be able to help triage the page into some kind of group, or some kind of pass system like you suggested. DannyH (WMF) (talk) 03:15, 1 June 2017 (UTC)

Having slept on this, and with a strong cup of coffee, if something like this is ever tried, it should probably be a passive, rather than an active feature, and whether the results are collected in a publicly available log/other area of high visibility should also be carefully considered.

I expect that if you force a reviewer to actively push a "pass" button, or have some comparatively highly visible this user passed on this article, we are likely to have that read in the minds of some, this user was too damned lazy to do anything about it, rather than this user did some level of evaluation, and if the article was obviously toxic probably would have taken some action, and at some level, enough "meh" probably amounts to the article not being an "important" part of the backlog, and instead just something that can just as well sit around for a few months before someone starts improving it.

If you don't build in some lower level of personal accountability (for lack of a better term), then all its probably going to end up doing is renaming a second "review" button, that's going to suffer from the same "badge of approval" problem we currently have. Instead, the goal would be more along the lines of reviewing being a sign of "does some action need to be taken", and sufficient "passes" an indication of "does some action need to be taken right now". TimothyJosephWood 12:36, 1 June 2017 (UTC)

Review of the report

The important thing to note is that adding more reviewers to this system will not make it work better.

This is true, and the community has not made a conscious drive to increase this number. However, the statement is true for the wrong reason(s). What does appear to be clear, although no one has been prepared to provide the stats, is that if each Reviewer had made only 50 reviews, the backlog would have long since been cleared. It’s caused by other patrollers who are not suitably experienced but who on the community's own insistence, are still allowed to tag pages.This naturally arouses the legitimate concern that some of the rights holders might possibly be hat collectors. Maintenance areas are a magnet to new and younger users - this is a fact of Internet life as anyone who has managed a busy web forum knows without having to spend months on dedicated research. The admins at WP:PERM are of course obliged to attach good faith to each application and only those that are clearly not suitable are declined. On this , I would invite Beeblebrox, Swarm, Juliancolton, Xaosflux, User:Mz7,Ymblanter, Lord Roem, and BU Rob13 to comment.

Backlog May 2017. Graph: Kudpung

• To be effective and present a realistic picture, the graph needs to go back to mid 2016 when the the backlog suddenly began to increase at an alarming rate, and for which everyone is apparently reluctant to provide a reason or even a theory.
  
• The WMF graph is also missing at least two other essential curves: The number of articles deleted, and the number of articles by new users that were deleted. The graph on the right shows the increase in the backlog since mid 2016.

While removing pages created by non-autoconfirmed users would reduce the burden on that first wave of reviewers, it would result in the loss of many potential good articles. It would also send a clear message to new Wikipedia editors that their contributions aren't wanted, potentially stunting the growth of the editing community.

This is based, apparently, on an opinion rather than on on fact, and nobody (from the volunteer community) has been suggesting pages created by non autoconfirmed users should be removed. If it's intended as an allusion to WP:ACTRIAL, then it's false. Indeed, It would not send such a message - in fact it would greatly reduce the number of totally inappropriate page creations (I won't even call them 'articles'). It would give good faith users time to read the instructions and polish up their articles to a state that would require much less intervention by other users who patrol and clean up; it would inspire other editors to provide help. We need to differentiate between regular editors who are happy to expand interesting potential articles, and the users who patrol the new pages to remove the unwanted ones and/or encourage the good faith creators to seek help. We cannot however, force our volunteers to do one, or the other, and certainly not both.

The top of the New Pages Feed says, in bold letters, Rather than speed, quality and depth of patrolling and the use of correct CSD criteria are essential to good reviewing.

The entire purpose of that piece of advice, which is misunderstood by the author(s) of the report, is specifically to avoid pages with potential, from good faith editors, from being flagged for deletion. This is one of the first major contradictions in the report.

A reviewer who doesn't spend enough quality time on a given review risks being blocked from reviewing...

This is conjecture and is inaccurate. The worst thing that can happen is that a New Page Reviewer can lose their flag and have to go back to patrolling without the convenience and advantages of the New Pages Feed and the right to mark pages as 'patroled'. As far as I know, of the 400+ reviewers, only two have had their flag removed and one of these was one who slipped through the grandfathering net. What does happen is, that very frequently newbies and other totally inexperienced users who are patrolling pages and getting it wrong are asked to refrain from tagging new pages until they have acquired sufficient experience. In these cases, the New Pages Feed serves as a double safety, and as far as I know, none have ever been blocked, and most of them were grateful for the advice they were given. We don't bite, but our tone may become decidedly less friendly if in spite of that advice the community has to issue a topic ban from patrolling new pages.

Summary

The urgent priorities are to:

• Continue to improve the New Pages Feed and its Curation Tool as requested at Wikipedia:Page Curation/Suggested improvements. This would ultimately make the system more appealing to New Page Reviewers who are still refusing to use it.
• Develop a proper landing page (please see this link) on the lines begun by Jorm for new users who are about to post a new page as their first edit. This would not discourage good faith editors. It would discourage spammers, vandals, hoaxers, etc., whose creations clearly make up the vast majority of new pages created by new users. The landing page was supposed to have been the other half of the new, New Page Patrol project. Instead, 6 years later we have an insurmountable backlog, a largely ineffective team of 400+ reviewers, and dozens of new users patrolling pages and scaring new, new article creators away.
• Combine the activities of AfC and NPR into one interface, for reasons not on this immediate agenda, but which DGG can probably explain better than I, but it would bring about a better utilisation of the recently created Draft namespace that was initially created as a place where new users could complete their articles withuot fear of rampant deletion before they are published in mainspace
• Prevent new and inexperienced users from doing certain maintenance tasks - this is not what was meant when the mantra The encyclopedia anyone can edit was coined.
• When using graphs to illustrate an argument, the full picture should be presented.
• It is not fully clear in the report that the difference between New Page Reviewers and ordinary patrollers is fully understood by the WMF team.

Conclusion

While I am pleased that the Foundation has finally decided to at least do some preliminary review of what is wrong with our page patrolling system, its taken a very long time, and I am sure that this progress has been achieved partly, though not entirely, through my constant whinging, lobbying, Skype conferences, and personal meetings with the WMF for a year with staff such as Danny Horn, Ryan Kaldari, MusikAnimal, Jonathan Morgan, Aaron Halfaker, Nick Wilson, and recently with Wes Moran (whose name has recently disappeared from the staff list). I would particularly like to thank Kaldari and MusikAnimal who have demonstrated the greatest understanding of the critical situation and have helped where they could within the limitations of their employment while dividing their time with their volunteer activities. Kudpung กุดผึ้ง (talk) 04:31, 1 June 2017 (UTC)

Discussion

• (moved from inline above)Regarding less experienced patrollers/hat collectors:Even if they make one valid patrol a year they helped the backlog. Not patrolling has them same affect as not being able to patrol, I don't see it as the cause of the backlog. However, if editors are making "bad" patrols that could be prevented by additional vetting before gaining access to the patrol tools, identifying additional granting guidelines can easily be incorporated to the permissions request system. — xaosflux ^Talk 11:03, 1 June 2017 (UTC)
• Going to create this section just to give us a place to discuss this. I agree with Xaosflux's inline comment above. Hat-collectors have a weakly positive effect – weakly in the sense that it may be zero, but positive in the sense that it cannot be negative. If we took away their shiny user rights, it wouldn't help. We have to face the reality that the problem is simple yet not easily fixed. The number of patrols that active editors are currently performing is exceeded by the number of articles being created. There's a high level of interest in improving the UI, talking about the problem, etc etc, but not much interest in devoting the actual manpower to fixing it. Or perhaps that manpower just doesn't exist. We need to stop talking so exclusively about the technical aspects of patrolling - who has the user right, the state of the UI, etc. - and focus on increasing the number of hours spent on patrolling. What are the segments of the editing community we haven't tapped into yet, and how could we more effectively get them on board? Chief among them are our content creators, who already have the most difficult-to-obtain skill required for patrolling, which is the knowledge of our content policies. ~ Rob₁₃^Talk 12:08, 1 June 2017 (UTC)
• I'd also like to separately urge admins to hand out the autopatrolled right whenever appropriate and editors to nominate others for autopatrolled (you can do that!) if you notice that someone's creating more than one article recently and qualifies for the user right. This helps the backlog in big ways! If we could get a bot to develop a list of editors who have created more than 10 articles in the past year, more than 25 total, and don't have autopatrolled (vs. the useless list of non-autopatrolled editors by total articles created, most of which are old editors), that would be helpful. ~ Rob₁₃^Talk 12:11, 1 June 2017 (UTC)
  • Rob, WMF has provided us with a report based on people who may qualify and have created an article in the last month Wikipedia:Database reports/Editors eligible for Autopatrol privilege. TonyBallioni (talk) 12:22, 1 June 2017 (UTC)
    • Hadn't gotten that far into this report yet. In two weeks, after I'm out of prelim hell (hopefully, anyway), I'll go through that database report and hand out autopatrolled as appropriate. ~ Rob₁₃^Talk 12:27, 1 June 2017 (UTC)
    • Please note, this is people that may qualify. — xaosflux ^Talk 12:56, 1 June 2017 (UTC)
      • Thanks for pointing that out. Yes, that is what I meant. Updated my wording. TonyBallioni (talk) 13:58, 1 June 2017 (UTC)
• Having had the time to reread the WMF report and everyone's analysis, here are the initial thoughts I have:

1. The graph raises more questions than answers and doesn't match the text of the article. It doesn't show deletions, which is a huge oversight. It also seems to suggest that approximately zero-ten of the total pages created by new editors are reviewed a day if you assume the daily average of 78 is correct and the backlogged articles on the graph are in the 65-75 range the first two days after creation. This doesn't seem right based on my experience, and I assume deletions have something to do with it.
2. The 7% number makes no sense compared to the graph. If the daily percentage of the backlog that is created by new users is in the 20% range, this would be the daily decrease in the backlog. Additionally, if 7% is the correct number of new pages created by new users and their pages do account for more than double that proportion of the backlog this is a major issue.
3. The idea that an arbitrary fall-off date could be helpful is misguided in my opinion and runs contrary to the WMF's stated strategic goal of being the most trusted source of knowledge by 2030. Even if just 10% of the pages in the fall-off group had serious issues that slipped by this would be a major issue for Wikipedia. This also ignores the fact that we are in fact patrolling the end of the backlog, and while it is growing, the dates are moving up at the end. Removing these pages would prevent review and improvement for years for most pages. This might not be an issue with the majority of pages on the end, but for the ones where it is an issue, it could be a major one.

I'm very appreciative of the WMF for paying attention to this and engaging with us, but the main suggestion in this proposal (the fall-off) is a very bad one in my opinion. The data also needs a second look over because it seems to not be easily explainable and is missing some key numbers. I hope these observations add some value to the conversation. TonyBallioni (talk) 16:36, 1 June 2017 (UTC)

@TonyBallioni: There are some misunderstandings here regarding the graph.

Re 1.: No, there is no "huge oversight" regarding deleted pages. As explained above, the graph represents a snapshot of the existing backlog based on exactly the same data as that underlying Special:NewPagesFeed. I.e. it counts the pages that are still unreviewed at a specific point in time (May 25), with the time axis corresponding to the creation date (or equivalently the page's age in the backlog). Deleted pages are of course not part of this backlog - they have already been dealt with.

Re 2.: It does not make sense to compare the 7% number to the ratios visible in the graph. The former refers to the total number of articles created, but the graph plots the number of articles created on that day that are still unreviewed - which, needless to say, is different. (And e.g. the fact that autopatrolled creations never enter the backlog means that the percentage of articles created by non-autoconfirmed users will be higher in the backlog right from the start, even before any patroller has had a chance to look at them.)

You are of course right that deletion rates are interesting too, and I'm looking forward to working with you on these on the Phabricator ticket. But they address different questions. In particular, they will provide information about quality differences between articles created by autoconfirmed and non-autoconfirmed users - I think nobody will be surprised if the latter get deleted more often. But that is not the same as asking which group is contributing more to the backlog, or which sucks up more patroller effort by generating time-consuming judgment calls. For that, the backlog graph is more informative.

Regards, Tbayer (WMF) (talk) 04:40, 2 June 2017 (UTC)

Non autoconfirmed new page creations

Wikipedia:New_pages_patrol/Analysis_and_proposal#Non-autoconfirmed_contributors appears to say that very few pages are created by non autoconfirmed users, but then it appears that autoconfirmed status is as measured now, not at the time of page creation. A quick look at the most recent creation tells me that non autoconfirmed new page creations are a lot higher than 7%. Autoconfirmation is a very low bar, and will be very easily met with a little fiddling of the first page. Just wondering about the facts. --SmokeyJoe (talk) 05:44, 1 June 2017 (UTC)

Well, after analyzing the graph further per the urging of Kudpung on my talk page, where I left a longer response I won't reproduce here, I have a question: where does the 7% number come from? By rough eyeballing of the first column in the graph it looks to be non-AC users account for ~75/275 of the daily backlog. That's 27%. That is a very significant number, especially considering how easy it is to reach AC. This is just eyeballing, so the actual percentage is likely less than that, but I would still but it in the 20% range from looking at it. TonyBallioni (talk) 06:34, 1 June 2017 (UTC)

There are a lot of things wrong with the graph and its interpretation. I'm hesitant to suggest it might be an attempt by the WMF to play down the crisis, but we have experts in data mining among the volunteer community who, given the actual requirement of what is needed to display the correct profile, could have done better. It is a great personal regret that I never learned to do these things and now it's too late. The correct data will certainly make some important changes to the focus of the report. Kudpung กุดผึ้ง (talk) 11:27, 1 June 2017 (UTC)

When it comes to promotional, sneaky promotional/POV articles, or WP:SNEAKY vandalism (either new articles, or in articles like Bay of Pigs invasion or Watergate Scandal); the perps seem to be aware of AC user-group. They usually wait for 5 days. Getting 5-10 edits in that time is not a big deal, they dont need to be article space either. Draft, userpage, talkpage; anything is good for that. I dont have the diffs to prove it, but i have observed it first handedly for a lot of times. Based on these observations, the conclusions/interpretations of the report/stats seem to be flawed. —usernamekiran(talk) 14:02, 1 June 2017 (UTC)

@SmokeyJoe and TonyBallioni: The 7% figure comes from https://phabricator.wikimedia.org/T149021#3287887. If you scroll down within the result, the full database query is provided. Let me see if I can explain this number a bit more and why it doesn't match the percentage of non-autoconfirmed articles in the backlog... 7% (or 6.57% if you want to be exact) is the percentage of new, main namespace pages that were created by non-autoconfirmed users from January 1, 2017 to April 30, 2017 that did not include the string "redir" or "Redir" in the edit summary (to try to screen out pages that were created as redirects). It includes pages that have since been deleted. This is the same query that gave us the "1,180 new articles created per day" stat. There are a couple reasons why this doesn't align with what you see in the backlog. First, the backlog excludes pages created by autopatrolled users. Second, the backlog includes pages that aren't technically "new", for example, pages moved into the main namespace from other namespaces like Draft and User, and pages that have been converted from redirects into articles. Unfortunately, it isn't possible to create a simple database query to match all of those cases, but you are correct that the percentage of pages entering the backlog that are created by non-autoconfirmed users is higher than 7%. The report itself says this in the second sentence of the Non-autoconfirmed contributors section. How much higher than 7%, we aren't sure. I think TonyBallioni's estimate of 20% is plausible though. One of the main points of the report, though, is that NPP is actually doing fine at patrolling articles by non-autoconfirmed users. Those pages tend to be handled pretty quickly (judging by the graph), especially the ones that are obviously bad. Hopefully we'll have better data soon on how many of those articles are actually getting deleted (T166269). Kaldari (talk) 21:24, 1 June 2017 (UTC)

I disagree with the analysis

Some parts of the the report are vague. At some instances, it is not clear if the report supports the reviewers/patrollers, or if it wants the right to be eliminated. Anyways. I mostly disagree with the section "This system is not sustainable". It says reviewers are generalists, this doesnt mean they dont have an area of expertise at all. If the reviewers are tripled in number, there are very high chances that at least one reviewer would be familiar with that particular field/category/subject with the backlogged article. Or there would be at least one editor from these 1000-1200 reviewers who would say "doesnt matter how long it would take, i will work on it". And in certain cases there is {{expert needed}} tag. I am pretty sure more than 95% reviewers know about these template as reviewer user-right is not granted easily.

"The only sustainable way to manage the backlog is to reinstitute the expiration date, which the system had from 2007 to 2012. An article that survives the gauntlet of reviewers for a reasonable amount of time – say, 30 days or 60 days – is unlikely to be picked up and fixed by a generalist new page reviewer. Pages that survive past that deadline should be improved by subject matter experts, which is the way that Wikipedia works. With a 30 day expiration, the backlog on May 30th, 2017 would have 5,650 pages instead of 21,800. With 60 day expiration, it would have 10,200 pages."

I think it is prohibited for subject matter experts to touch the page if it is unreviewed. Maybe they get blocked for 24 hours for just clicking the "edit" on an unreviewed page. Why not set the expiration date at 10 days? It will be really good for everybody. This will be exactly like, changing the definition/upper limit of high blood pressure to be able to say "no! The parient doesnt have a high blood pressure." —usernamekiran(talk) 13:50, June 1, 2017 (UTC)

How does adding an expiration date supposedly "fix" anything? Yes the backlog stops reporting so many numbers , but it doesn't change the quality or content of the new articles and doesn't impact readers or editors. We already have a 30 day INDEXING expiration - and that is something that is actually reader impacting. @Kudpung: please tell me I'm not missing something here! — xaosflux ^Talk 14:45, 1 June 2017 (UTC)

I absolutely support the reviewers/patrollers, who are doing really important work. What I'm saying is that the system that's evolved is unsustainable, for two reasons: the standard for an acceptable page/review is getting higher, and we're not allowing good pages to age out of the backlog naturally. That means that people are spending a lot of time improving pages that are already pretty good, rather than clearing out bad pages and allowing the pretty good ones to age out. The other side of the high blood pressure analogy is that the patient doesn't want to change any of the behaviors that led to having high blood pressure.

Xaosflux: the point of an expiration date is that it encourages reviewers to actually make decisions, rather than looking at the page, making an edit, and not marking it as reviewed. There are a lot of pages that have been looked at and even improved by multiple reviewers, but they're still in the backlog because there's no consequence to just leaving it there. In my opinion, that system will always create a backlog. DannyH (WMF) (talk) 15:15, 1 June 2017 (UTC)

@Xaosflux: yes, thats my exact point. I apologise for using sarcstic/rhetorical words, sorry for the confusion. I meant, not looking at something doesnt make it go away from existence. It still remains there. Thats what happens with the backlog, even if the expiration dates are changed, the bad articles which were not looked at will remain there. They are not going to change themselves in good articles; the copyrighted content is going remain copyrighted, promotional content is going to remain promotional no matter if the article is the backlogged group or not.

@DannyH (WMF): yes, I partially agree with you. The basic point of everything on wikipedia is, we should work towards creating good articles. Then, whether it is backlogged or not becomes sort of immaterial. If there are active reviewers, then at the least, the "easy ones" will get marked as patrolled/reviewed. That would in turn, result only the articles with "Time Consuming Judgment Calls" (TCJC). These articles can later be handled by the editors of that particular field. After weeding out the "easy ones", I dont think handling the articles with TCJC would be a difficult matter. (Yay! I just coined a new term lol).

In conclusion, we need reviewers who are active, and a good strategy. Strategy is the key here. —usernamekiran(talk) 15:42, 1 June 2017 (UTC)

(edit conflict) Yes, the point of an expiration date is that it encourages reviewers to actually make decisions, rather than looking at the page, making an edit, and not marking it as reviewed, DannyH (WMF), but I doubt that the rest of the first part of your comment is in any way founded on facts. I would point out (and for Xaosflux), that NO_INDEX actually expires after 90 days (and fortunately so) . And here's how it was done, by Kaldari, Roan Kattouw (WMF), Cenarium, and myself, in a perfect example of how true Community/WMF collaboration should ideally take place.

Since I stepped back in February from 6 years of actively campaigning for improvement in the way we police new pages, I have gone back to spending sometimes up to 14 hours a day reviewing new pages. Don't get me wrong - I'm not doing it specially to help reduce the backlog, although of course that happens; I'm doing it to do that all-important empirical study. The standard for an acceptable page/review is not getting higher. In fact it's pretty much the same as it ever was and this does not explain the backlog at all. Reviewers are not spending any more time on their reviews than they did before. To be sure of your facts you need to check on the Curation logs of every one of the New Page Reviewers and see how long they took over each review during a session.

There needs to be a clear understanding of the distinction between New Page Patrollers and New Page Reviewers. They are both very different animals. Partly the reason that the system is unsustainable is because Community Tech blocked any development that would have made New Pages Feed and its Curation Tool the perfect alternative to Twinkle, and partly because the community itself insisted on allowing MMORP gamers to continue to mess with New Page Patrol. Those of us like DGG and me who patrol new pages to gain an overview of the bigger picture, spend half our time chasing the newbies away from this important and complex task. I fail to understand why the Foundation would support a system that allows inexperienced users to bite good faith creators of new pages on the one hand, while suggesting that spam, hoaxes, attacks, and other rubbish should be allowed to 'to age out' for simply not having been patrolled within the time limit. Kudpung กุดผึ้ง (talk) 16:19, 1 June 2017 (UTC)

@DannyH (WMF): As a benefit of my career, I've gotten somewhat adept at differentiating between motivations that actually drive behavior and good-on-paper motivations that don't. Your suggested behavior pattern for editors sounds plausible, but I highly doubt our reviewers think about the overall impact on the backlog of their individual actions, since their individual actions form such a small part of the aggregate. I can't see any plausible method to shift from "If I don't do this review and spend time on another, someone else will just do this" to "If I don't do this, no-one will". If anything, we'd stress out people working on the backlog from oldest to newest, who would see things going by without any ability to really stop it. That could cause burn-out due to a sense of not accomplishing anything. If you want to say we should impose an expiration date because the content remaining in the queue is a time-sink that can be improved just as well organically, then I disagree, but that's a matter of opinion. If you're saying we should impose an expiration date because it will drive a higher quantity or quality of reviews, I think that's fundamentally incorrect. There's no evidence to support that. ~ Rob₁₃^Talk 00:12, 2 June 2017 (UTC)

Some Background

There seems to be a popular misconception that ACTRIAL was only about preventing non confirmed users from creating pages. That perception is completely wrong and discredits those who worked hard to create the ACTRIAL project and the hundreds of users who voted the overwhelming consensus for it. In 2006 by withdrawing the 'right' of IPs to create pages, the Foundation already acknowledged that Wikipedia is organic and that the rules occasionally need to be modified accordingly.

It is erroneous to allow the impression that the current backlog began concomitant with the creation of the New Page Reviewer user right in November. It didn't. This current backlog actually stretches back to mid 2016 (where it was 'only' 5,000). This was a already a grave concern and is what gave rise to the talks in Italy and the run up to the creation of the New Page Reviewer group in November. In fact for a while, until it suddenly started rising dramatically again in February, after the roll out the backlog actually began slowly but surely to diminish.

The Foundation has now given us a page full of comment, which is genuinely very welcome and highly encouraging and I'm sure that those commenting here will read it entirely and carefully, but in order to get properly up to speed, the WMF team should probably also be encouraged to do the community team the courtesy of reading the whole of this page: Wikipedia talk:The future of NPP and AfC. Kudpung กุดผึ้ง (talk) 15:10, 1 June 2017 (UTC)

Does this mean, "After 30 days, any pending submission passes by default"?

• Update: DISREGARD EVERYTHING I SAID. I had no idea what was going on. Blue Rasberry (talk) 23:59, 1 June 2017 (UTC)

There is a lot of information here and I am not following this proposal. Suppose that someone makes a submission and it gets no review after 30 days. After that point, will it move into Wikipedia mainspace in the same way as an article which passed review? Blue Rasberry (talk) 20:08, 1 June 2017 (UTC)

@Blue Rasberry: Not yet. But yes, it is being thought i think. "To help decrease the backlog" apparently. —usernamekiran(talk) 20:39, 1 June 2017 (UTC)

• previously pinged incorrect username. Bluerasberry —usernamekiran(talk) 20:40, 1 June 2017 (UTC)

I would like to see a random sample of proposed "default pass" AfC submissions go to AfD. I have imagined that the bar for passing AfC is the same as passing AfD. I do not see data to back the claim that the default for not getting a review should be "pass" instead of "delete". Deleting anything which does not pass a review after a certain amount of time is an option also. In AfD if something does not clearly pass, then people vote delete. The current AfC process does not currently default to delete in that way because it is intended to be friendlier to new users.

In this proposal, what I am seeing is the idea that multiple reviewers pass problem cases. I want those out of the queue to make way for better review, and I am not convinced that marginal cases are better passed than deleted. If marginal cases are 50% passing quality and 50% deleting quality, then I might say delete. Some of this proposal seems to suggest that marginal cases are 90%+ passing, which if that is the case, I can understand passing them all by default. If the numbers are 50% or lower, I say delete by default.

I wonder if changing the structure to pass by default will change the stability of tendency to pass/delete article submissions.

Overall, I still do not understand the data presented here or what it means. Blue Rasberry (talk) 20:56, 1 June 2017 (UTC)

I'm... confused. Where does AfC come into this? TimothyJosephWood 21:11, 1 June 2017 (UTC)

I think you mean "AfD". My claim was that the AfC process should never move a draft into the mainspace if that draft would immediately fail AfD. I expect that most people would oppose a plan to move articles likely to fail AfD into the mainspace. How do you feel about having articles likely to fail AfD in the mainspace? Blue Rasberry (talk) 23:50, 1 June 2017 (UTC)

Hi @Bluerasberry: The ultimate decisions about how the system should work are up to the New Pages Patrol team, and the volunteer community as a whole. The report that we posted is our view of what's causing the current situation, and recommendations for changes that NPP can make. It's meant as a step in a larger NPP/Foundation/community conversation.

The fact that there's a 22,000 page backlog means that there's something that's not working with this system. At the moment, the default setting is de facto "pass", because all of those pages are currently live on Wikipedia. As far as readers and editors are concerned, whether an article is marked as reviewed or not is invisible. The pages are part of the encyclopedia.

As I wrote in the proposal, I think that the main drivers of the huge backlog are the increasingly high standards for what constitutes a reviewed page, which changed in September 2015, and the tighter control over how fast reviewers are going, which changed in November 2016. The system is asking people to go slower and spend more time with each review, and encourages people to pass and move on to another page. That means reviewers pass on the most time-consuming pages, and those are the pages that end up in the backlog. Telling reviewers "just go make 50 reviews" is too much to ask, if each review could take days to complete. There needs to be something in the system that acts as a motivator for people to actually make decisions, whether that's pass or delete.

For the Foundation, we're very interested in helping this system to work better, but we don't want to invest resources in a system that we think is fundamentally unsustainable. We can't make the decisions about how the system works, but we want to be involved in the conversation. I hope that makes more sense? Let me know if I'm still being unclear. DannyH (WMF) (talk) 21:30, 1 June 2017 (UTC)

@DannyH (WMF):What do you mean by "all of those pages are currently live on Wikipedia"? Do you mean "live" as in "draft space"? I would not call that live, so clarify if that is what you mean. If that is the kind of default you mean - "still in draft space, but no longer in the review queue", then I was not understanding that. Blue Rasberry (talk) 23:48, 1 June 2017 (UTC)

Bluerasberry, the solution that you and Chris Troutman suggest (nominating all articles older than X days for deletion) is already technically possible. A bot could easily handle that. It might be worth seeing if there is broader consensus for implementing that as a solution. Kaldari (talk) 21:50, 1 June 2017 (UTC)

@Kaldari: nope, it would be a disastrous move. Plenty of good articles would get deleted just because a group of limited users couldnt process it. And bots are dumb after-all. Isnt that right, DumbBOT? (Just nod your head if you want more electricity.)

It is easy to foresee this solution will never get the consensus. —usernamekiran(talk) 22:08, 1 June 2017 (UTC)

It is easy to foresee this solution will never get the consensus. That's probably the one thing we can all agree on. TimothyJosephWood 23:27, 1 June 2017 (UTC)

@Timothyjosephwood and Usernamekiran: Can you restate your view? There are lots of article in limbo in draft space. The backlog is 20,000. Options for addressing this include leaving them in limbo/draft space perpetually, or deleting them after some time, or moving them from draft space to mainspace after some time. If I understand correctly, you are saying that there will never be consensus to delete draftspace articles after some time. Do either of you believe that there can ever be consensus to pass these articles from draftspace into mainspace by default after some time? Blue Rasberry (talk) 23:48, 1 June 2017 (UTC)

@Bluerasberry: All of the pages that we're talking about are already in mainspace; that's the 22,000 page backlog. Drafts/AfC is a different workflow. DannyH (WMF) (talk) 23:50, 1 June 2017 (UTC)

(edit conflict)@Bluerasberry: Erm... I think you were confused right before you posted your first comment today. We are talking about the articles that are already published, and need to be reviewed. —usernamekiran(talk) 23:55, 1 June 2017 (UTC)

Thanks, I was confused. Blue Rasberry (talk) 23:59, 1 June 2017 (UTC)

The WMF's solution is really a head-scratcher to me. They think the problem is "The queue is too big" so the solution is "Just throw stuff out of the queue after 30 days". If the only metric we're going by is length of queue, I have an even better solution. Get rid of the queue entirely and the queue length will be zero! The secondary metric (quality of content) is lacking in this report. Further, they fundamentally misunderstand the guidance given to patrollers. They wrote "Following the Article namespace checklist – the minimum effort that a reviewer is supposed to do – this article would probably take days to fix. You'd have to track down references, most of them not in English, and completely rewrite the page from scratch." That's incorrect. The guidelines for patrollers say to fix easy issues and tag more complicated things, which doesn't take long at all. Same goes for notability. If notability is seriously questionable, tag with questionable notability (or bring it to AfD for more opinions, which is also fine). Detecting issues and fixing issues are very different things with very different time commitments, and so the analysis is flawed. If we have a messaging problem where patrollers think we're asking them to make every article GA-quality, let's talk about that. But if every reviewer followed the guidance given to them, there's no reason to believe articles in need of improvement are driving the backlog. ~ Rob₁₃^Talk 00:06, 2 June 2017 (UTC)

if every reviewer followed the guidance given to them But they're not. So we can talk about a perfect world all day, but how do we square the de jure with the de facto in a way that in a reasonable logistical sense accomplishes the mission? TimothyJosephWood 00:15, 2 June 2017 (UTC)

I'm just going to add a touch of a rant so forgive me. I've seen if only every reviewer would review X amount of articles the system would work so many times it's starting to make me nauseous. They're not...we're not and we have to ignore the idea that it's a problem with the reviewers because that's not a problem we can solve. So what problem can we solve? TimothyJosephWood 00:20, 2 June 2017 (UTC)

Well, until we see the new numbers the WMF is working on for deletions and backlog by autoconfirmed status at the time of creation (not today) I don't think we know much. Like Chris troutman and I suspect others, I haven't seen anything that has convinced me that pulling the trigger on ACTRIAL wouldn't help. Others have also suggested in the past switching the default place for page creation to be draft space, which would shift the burden. What I am confident in is that the obscuring of the number of unreviewed pages by having a cutoff isn't the solution. TonyBallioni (talk) 00:29, 2 June 2017 (UTC)

• It seems quite clear (to me at least) that either now they've started the dialogue, the Foundation is not fully paying attention to what is being posted here by the community, or the Foundation team has never actually spent any significant time at the coal face - or both. It also seems to me, and I sincerely hope I'm wrong, that they are belittling the huge efforts made by a few concerned editors over the past 6 years to to get New Page Patrolling cleaned up and some genuine good faith attempts to bring some modernisation into into the way new pages (I won't say 'articles') are created and by whom.

We never had an explanation either as to why Jorm's development of a proper landing page was quietly, but deliberately shelved and hidden behind a wall of multiple page moves. Jorm won't tell us why, not even me when we met privately in 2014 (possibly because he's muzzled by some non-disclosure clause). Let's not forget that the lack of a proper landing page is half the current problem.

And all the while I am very curious why DannyH (WMF), who incidentally personally blocked further development on the Page Curation system by insisting that this critical issue join a community wish list, until suddenly opening this dialogue, is still insisting that this backlog is a recent phenomenon that dates back to when we rolled out the New Page Reviewer right. The table I posted above should be clear enough. I was politely given the brush off already in June last year in Esino Lario by a WMF staff when I tried - in the presence of other en.Wiki admins - to discuss the situation, and a 3-hour private discussion on a hotel terrace with another one during the same conference was constantly met with 'rest assured that the WMF is actively doing its best to resolve these issues' (or words to that effect) - these are the kind of comments we're used to hearing from British Prime Ministers. In the corridor of GWU in D.C. back in 2012 when we attempted to broach the issue we were actually told very rudely to F*** off, by a contractor, and his boss, a C-Leveler, told us to mind our own business. I don't like conspiracy theories, but what on earth is going on? Kudpung กุดผึ้ง (talk) 01:23, 2 June 2017 (UTC)

There's no conspiracy. Jorm's Article Creation Workflow was killed for two reasons: The community was moving towards creating a Drafts namespace (which would have required reworking the workflow), and the WMF decided that working on the Echo extension (i.e. notifications) was higher priority. The engineers that were working on Article Creation Workflow and Page Curation were simply re-assigned to work on Echo. Luckily we were able to (mostly) finish Page Curation and launch it (but only for English Wikipedia). Article Creation Workflow was never more than halfway finished, and at this point it would need to be redone from scratch. Kaldari (talk) 02:11, 2 June 2017 (UTC)

What is wrong

It is completely wron that "The only sustainable way to manage the backlog is to reinstitute the expiration date" The effect of a cut off date will be to keep articles from ever being reviewed. There is no reason why anything at all should totally drop off the end--if there is any purpose to reviewing, everything needs review. All that having an artificial cut off does it prevents us from realizing the extent of the backlog, by deceiving us into not seeing it. Having such a date is the basic recommendation of the report, and it is absolutely counterproductive, an admission of defeat. It is the reaction of bureaucrats who want to pretend they have control of a problem, not editors who know the limitation of Wikipedia. It will indeed make the system look better. Bureaucrats and system professionals care about this, they do not really care it it fulfills the function. They just want it to look professional. (This is not meant as personal--when i was a professional in a complex system, it was most important to me that my library appeared excellent --I was very aware of the things that were grossly imperfect but that I could not affect, and I did very well at making them invisible.)

It is completely wrong that "adding more reviewers to this system will not make it work better." The most important thing to note is the direct opposite: without adding more reviewers to the system , it will never work much better. The only real way of improving anything at WP is more participation in the process. The principal goal of us all should be to increase participation at every step--starting with people reading articles being willing to make obvious improvements, all the way through every step in increased involvement, all the way to being regularly writing new articles. If retention at each of the many steps were increase even slightly, the overall effect would be significant. When we look at the details, our goal should mostly be to remove impediments. We do not need to come from outside to design a system. The entire principle underlying the Wikipedia projects is that the system is self-designing and self-correcting. Not everything can be done by such methods; Wikipedia is not the all-encompassing intellectual product of mankind, but has a special role: a general purpose encyclopedia of first resort universally available. What can be done by amateurs working with informal coordination is what we should do. What requires specialists or professionals or centralization is what should not be part of the projects, and was never from the first intended to be.

It does not take expertise to do most reviewing--neither great expertise at WP and certainly not expertise in the subject. That doesn't mean one person can do everything, but as a person does get experience, they can move to the more difficult articles. Reviewing is meant to be a first pass, and part of the problem is that its function and role has become overloaded to the point where it becomes an impediment. It should not be reviewing in detail or definitively; if it were, it would indeed be impossible to keep up. Rather, the point of reviewing is a first pass, to mark the things that must be removed, and to indicate some of the key problems. Articles are further assessed continually as people see and work on them. Each of us does have subject limitations, but again , the basic principle of a project with widespread participation, is that among us all, we will cover all the fields. What cannot be done this way, is unsuitable for us to attempt. We've seen in the development of WP, a wider and wider expansion of what volunteers are able to do and want to do. It has not been centrally developed.

Given the existing or attainable levels and types of participation, we do not primarily need improved technologies of review--I and I think most good reviewers almost never use the reviewing toolbar except for its convenient functionality in scanning. (I do use twinkle--this is an example of a combination of locally developed stopgap methods whose usefulness has been greatly expanded by widespread adoption, rather than something actually planned from the first.) What we do need is for most experienced WPedians regardless of primary interest to look at a small number of new articles each day,as part of their normal participation here. Accepting the figure of 1200 articles a day, it will be better if 200 people each review 6, than if 20 people each review 60.

There is a genuine but limited role for professionals at WP: to devise tools that volunteers seem not to want to deal with. There are two tools we need (undoubtedly others can suggest additional ones): a prescreening for likely copyvio at the time of submission, and a rough system of subject classification at input. The available technology can do these. So can AI, but we need not wait for that. Where AI might be useful is in distinguish promotional edits, where we cannot yet explicitly specify how we are judging.

Nothing about this is actually broken, in the sense of not working. Many things are not working very well, and if WP is to do what WP can do, that will always be the case. Our role is to work at the frontiers of what volunteers can accomplish. Things will always be rather rough out there. It's supposed to be that way. It's for doing this sort of unpredictable work that we need our sort of project. (I want to emphasise that I know individualy about half the people contributing to this report--based on what I know of them, any one of them could have done more realistically by themselves,and all of them do more realistically in their volunteer capacities.) DGG ( talk ) 03:37, 2 June 2017 (UTC)