Wikipedia:Bots/Requests for approval/28bot 4

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was

Approved.

28bot 4

Operator: 28bytes (talk · contribs)

Time filed: 21:22, Monday February 27, 2012 (UTC)

Automatic, Supervised, or Manual: Automatic

Programming language(s): Python/Pywikipedia

Source code available: N/A

Function overview: (a) Revert the insertion of a ~~~~ signature in an article, and (b) revert edits where two or more edit tests are found.

Links to relevant discussions (where appropriate): Village Pump discussion right before it was archived

Edit period(s): Continuous

Estimated number of pages affected: between 10–40 per day

Exclusion compliant (Y/N): ~~I plan to add exclusion compliance to the code if this task is approved.~~ Y: Exclusion compliance added in build 39, March 11, 2012. Respects {{bots}} template in both article space and on users' talk pages.

Already has a bot flag (Y/N): Y

Function details: (a) Revert edits which insert a ~~~~ signature in an article, and (b) revert edits where two or more edit tests are found. Examples of such edits are included in the related Village Pump discussion. 28bytes (talk) 21:22, 27 February 2012 (UTC)[reply]

Discussion

Revert in what way? With what message? Wouldn't it be better to simply have a disallow edit filter? - Jarry1250 ^{[Deliberation needed]} 23:02, 27 February 2012 (UTC)[reply]

Revert like this, I suppose, or am I misunderstanding the question? I don't do edit filters anymore, but if someone else wants to write one, that's fine by me. For the message I was thinking Template:Uw-articlesig. 28bytes (talk) 23:34, 27 February 2012 (UTC)[reply]

For the record: I expect the bot to only edit in article space.

It's been through the appropriate forum. Fears were raised that incompetent good faith edits would be lost rather than corrected. I want the bot to keep a central log for each of both (a) and (b) of all the additions that were made, so humans can subsequently quickly review to see if there's an acceptable rate of reverting incompetent good faith (vs test) edits. Include appropriate audit information.

It will also make reviewing the trial much easier.

Existing trusted op, non-contentious well publicized task, bot coded and tested already, so

Approved for trial (200 edits or 5 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete. whichever comes first. Advise here of any faults detected. A trial of this size will give the community suffient information to decide if automatic reverting is harmful. Josh Parris 00:34, 28 February 2012 (UTC)[reply]

Thanks Josh. Yes, it's currently coded to only look at (1) article space and (2) a couple of pages in my userspace. Logging the reverts will be no problem, and I'll list any false positives here. 28bytes (talk) 00:51, 28 February 2012 (UTC)[reply]

Note to self: publicize results for community reflection. Josh Parris 07:00, 1 March 2012 (UTC)[reply]

Can the trial be extended to March 10? The new functionality's only been in place since yesterday and the bot's not found anything to revert yet. FYI, when it does find something to revert, it will log an entry here: "Reverting TYPE 5 edit tests" for IP signatures and "Reverting TYPE 10 edit tests" for multiple edit tests. (The "TYPE 0" reverts are for edit test reverts approved in the first BRFA; that functionality hasn't changed and is still at 0% false positives.) 28bytes (talk) 00:28, 5 March 2012 (UTC)[reply]

Approved for extended trial. Please provide a link to the relevant contributions and/or diffs when the trial is complete. until the above-mentioned date, sure. Ask for another extension if need be. Josh Parris 05:41, 6 March 2012 (UTC)[reply]

Thanks. 28bytes (talk) 13:23, 6 March 2012 (UTC)[reply]

Trial results

Trial complete. Here are the details of what the bot has reverted so far. For the IP signature edits (part "a" of this BRFA), the bot has made the following reverts that I would call unquestionably good, as the edit reverted was obvious vandalism or edit tests with no redeeming changes: [1] [2] [3] [4] [5] [6] (the bot attempted to revert but was beaten to it by another editor) [7] [8] [9] [10] [11] [12] [13] ~~[14]~~ ~~[15]~~ [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29]

The following I would classify as "grey areas": while the edit was correctly (IMO) reverted, there were attempts at improving the article along with the misplaced signature: [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] (the bot happened to revert the addition of a giant copyvio here) [40] [41]

For the reverts of multiple edit tests made in one edit (part "b" of this BRFA) reverts, I would characterize these as clearly good reverts: [42] [43] [44] [45] [46] (attempted revert, beaten by ClueBot) [47] [48] [49] [50] [51] [52] [53] [54] [55]

As with the IP signature reverts, there were a couple of grey areas. I wouldn't call them false positives (as I would have manually undone such edits myself if those articles had been on my watchlist) but they don't appear to be vandalism: [56] [57] [58]

Of all the reverts the bot made during the trial, this is the only one that, IMO, might have been better as a partial revert than a complete revert: [59]. 28bytes (talk) 20:54, 10 March 2012 (UTC)[reply]

To summarize: you think all reverts are defensible, and the heuristic to identify dud edits has held up well.

You've listed http://en.wikipedia.org/w/index.php?title=Mako:_Island_of_Secrets&diff=prev&oldid=480749176 as both "unquestionably good" and "may have been better as a partial revert" and "grey area".

You've listed http://en.wikipedia.org/w/index.php?title=Twyla_Tharp&diff=prev&oldid=480693647 as both "unquestionably good" and "grey area".

Sometimes there's as much as a 20min delay between the edit being made and the revert; why?

Bearing in mind Wikipedia:Competency is required, and given there are such a large proportion of "grey area" edits, where there was an incompetent attempt to improve the article, do you still feel the basis of this BRfA is solid?

There are only 57 edits listed above. I think we may need a larger sample to make a call; what about you? Josh Parris 23:52, 10 March 2012 (UTC)[reply]

Oops, meant to put [60] and [61] solely in the "grey area" column. From an editorial point of view the addition of "Twyla Tharp is still living today and loves to dance for others" is not a helpful addition to the article (more so that it was inserted in between the categories and iw links), but it's certainly not vandalism. I intended to put all of the good faith edits, no matter how ill-conceived or poorly executed, in the "grey area" column. For the Mako article, the editor added an IP signature at the top of the page (bad), added an Oxford comma (unnecessary but harmless) and changed the capitalization of "Moon" to "moon" (good, since that's how all three sources have it.) I've reinstated the capitalization change manually. Unlike most of the other IP signature insertions, where just removing the signature would leave a variety of other unwelcome junk in the article, in this case a partial revert would have been ideal, although the full revert still leaves the article in better shape than no revert at all, IMO.

The 20 minute delay is an artifact of the way it processes the recent changes list; I've got on my "to do list" to rework that logic to hopefully cut down the delay. Based on the results so far I am satisfied that all of the bot's reverts have improved the encyclopedia, but I am happy to run it for longer and collect more data. 28bytes (talk) 02:18, 11 March 2012 (UTC)[reply]

In future logs please include a link to a diff.

It occurs to me that messaging editors may lead to good-faith material being re-inserted, or at least leave you a defensible position after removal; what do you think of that idea?

Do you know (or suspect) why the vandalism edits weren't caught by the anti-vandal bots? Josh Parris 10:52, 11 March 2012 (UTC)[reply]

I'll add diff-logging to the next build. Regarding messaging, it's currently configured to post User:28bot/templates/type5 for IP signatures and one of the first four listed at User:28bot/edit-tests-found#Templates for multiple edit tests (depending on whether the user is registered or unregistered, etc.) I have no idea why edits like the ones reverted in [62], [63] and especially [64], for example, weren't caught by the edit filters, ClueBot, RCP, or anything else until my bot came along. For the second one, perhaps the presence of a <ref> tag had something to do with it. 28bytes (talk) 18:40, 11 March 2012 (UTC)[reply]

Diff-logging has been added. 28bytes (talk) 02:04, 12 March 2012 (UTC)[reply]

Have you considered any metrics regarding the effect of the message you've been posting? Josh Parris 08:44, 12 March 2012 (UTC)[reply]

The WMF was doing some of that here; I'm certainly open to suggestions for additional metrics. 28bytes (talk) 03:32, 13 March 2012 (UTC)[reply]

Second trial

I'm asking for a second trial because the edit period of the first only generated 56 changes, of which 14 were type b) - I don't feel this is enough for the community to make a statistically valid call; past approvals for bots have required very low error rates for broad-based bot reverting. I'm comfortable asking for such a large second trial because it is clear the operator is very closely monitoring the bot during its trials, so I expect that no heinous behaviour will occur. I estimate the trial will run somewhere in the area of six weeks. Approved for extended trial (250 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. (weekly-ish counts - or something - as a stay-alive on the BRfA please) Josh Parris 08:44, 12 March 2012 (UTC)[reply]

Will do. Found my first actual false positive yesterday; an editor added a bunch of valid text and a bunch of <big></big> tags to an article they were building. I undid the bot's revert, cleaned out the extraneous tags manually and have updated the bot to ignore <big></big> tags. The article was IGLOO (Band) (since deleted). 28bytes (talk) 03:38, 13 March 2012 (UTC)[reply]

Just a little note, I thought this task was already handled by this bot? Am I right? Ceradon ^talk_contribs 23:14, 13 March 2012 (UTC)[reply]

Wikipedia:Bots/Requests for approval/RscprinterBot is supervised, this bot is unsupervised.

I don't have any problem with duplicated functionality; operators retire, go on wikibreaks and have their operating platform fail, so a broad ecosystem of bots is desirable. And different implementations have different algorithms, leading to bots that have the same top-level functionality but exhibit different behaviour. Josh Parris 23:26, 13 March 2012 (UTC)[reply]

I agree, redundancy is a good thing. However, I feel compelled to note that 28bot has been reverting edit tests since mid-2011; RscprinterBot is a more recent addition to the bot arsenal. The original (I think) bot that did this was Wikipedia:Bots/Requests for approval/AmeliorationBot, but its operator is no longer active. 28bytes (talk) 23:39, 13 March 2012 (UTC)[reply]

That is correct, I was mainly inspired to give RscprinterBot its task because of the redundancy of AmeliorationBot which had stopped editing. I felt the task was important as some signatures from good-faithed IPs are included in constructive edits, therefore slipping through the net of outright vandalism patrolled by ClueBot NG, but if no bot is cleaning up it must wait until a human finds it, which could be a while. So, although this task is duplicated, there is no harm done in having another bot do it as well. In our 'bot arsenal', as it were. Rcsprinter (talk to me) 16:37, 14 March 2012 (UTC)[reply]

Update

Since the last round of diffs, there have been 31 reverts of IP signatures. Of those, only 1 appeared to be a mixed edit (i.e. potentially helpful content mixed in with the signature): [65]. The editor subsequently made two more edits adding content to the page, both of which were reverted by another IP as being factually wrong (none of the edits were referenced).

There have been 11 reverts of multiple test edits; 1 was the false positive discussed above that has since been fixed, the other 10 were valid. 28bytes (talk) 20:15, 17 March 2012 (UTC)[reply]

Update 2

Since last week, 18 IP signature edits reverted and 6 multiple edit test edits reverted. No false positives, although a couple of the edits reverted were terribly-executed attempts to do something useful.

One interesting thing that happened: someone removed some vandalism from an article; ClueBot reverted them, reinserting the vandalism and warning the editor; 28bot then reverted ClueBot and removed the vandalism again. The IP who removed the vandalism also removed an image, which was probably a mistake, and probably why ClueBot generated the false positive; I restored the image manually (and removed ClueBot's warning message to the IP). I will continue to monitor. 28bytes (talk) 06:27, 24 March 2012 (UTC)[reply]

Update 3

19 reverts of IP signatures and 4 reverts of multiple edit tests this week. No false positives, although I did get one question on the bot's talk page about this edit being reverted. Apparently the editor was trying to restore a missing paragraph from the page history and inadvertently re-introduced a bunch of old vandalism. The bot noticed the vandalism and reverted the edit, which it was correct to do. I manually restored the paragraph the editor was trying to put back. 28bytes (talk) 08:23, 31 March 2012 (UTC)[reply]

Update 4

This week's stats: 36 reverts of IP signatures, 10 reverts of multiple edit tests. No false positives. 28bytes (talk) 03:36, 8 April 2012 (UTC)[reply]

Update 5

This week's stats: 38 reverts of IP signatures, 13 reverts of multiple edit tests. One "mixed" edit reverted. 28bytes (talk) 22:10, 13 April 2012 (UTC)[reply]

Update 6

This week's stats: 54 reverts of IP signatures, 8 reverts of multiple edit tests. Two "mixed" edit reverted; in both cases editors reinserted the intended text without the accompanying junk. (Which is exactly what the bot asked them to do, so everything's still working as designed.) 28bytes (talk) 03:55, 21 April 2012 (UTC)[reply]

Update 7

This week's stats: 61 reverts of IP signatures, 15 reverts of multiple edit tests, no false positives. 28bytes (talk) 20:51, 28 April 2012 (UTC)[reply]

Trial complete. Looks like with this week's work it's completed the 250 edits requested for the extended trial. 28bytes (talk) 20:55, 28 April 2012 (UTC)[reply]

Approved. MBisanz ^talk 23:45, 2 May 2012 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.