Wikipedia:Wikipedia Signpost/2013-07-03/Technology report

Technology report

VisualEditor in midst of game-changing deployment series

WYSIWYG editors, 2004 to 2011

Wikipedia:Wikipedia Signpost/Rquote Wikipedia:Wikipedia Signpost/Rquote The idea of a fully-integrated What-You-See-Is-What-You-Get (WYSIWYG) editor has a long history. The associated Meta-wiki page—now a redirect—was created in 2004, drawing on an already significant literature. Much of its content then ("many would-be users of MediaWiki are put off by what looks to them—rightly—to be code of any sort") would not look out of place in a discussion of contemporary developments, and at least one of the page's authors, Gabriel Wicke, has been able to help bring his ideas into fruition as a staff developer. When asked in July 2004 what one thing he'd change about Wikipedia, wiki-innovator Ward Cunningham immediately listed the installation of a WYSIWYG editor.

With much of the web moving its user-facing interfaces over to various WYSIWYG systems in the years following (Wordpress went WYSIWYG in December 2005, for example), several WYSIWYG editors—specially tailored to output Wikitext rather than HTML—were created, including a popular fork of FCKEditor (2007). Commercial wikifarms Seedwiki and Wikia were among those to benefit, allowing site administrators to banish wikitext from their wikis as early as 2005.

Nevertheless, the freedom that Wikitext provides editors, the difficulties of working with MediaWiki's evolving and generally idiosyncratic Wikitext-to-HTML parser, and a drive for perfection prevented the introduction of any WYSIWYG editing capability to Wikimedia wikis—at least by default (previous Signpost coverage).

WMF development, 2011 to present

That changed in May 2011, when the Wikimedia Foundation took on the challenge of creating a fully functioning visual editor (a commitment with its origins in the 2010 Wikimedia Usability Initiative and confirmed in 2011's product whitepaper, which described rich-text editing as a "great movement project").

At the time, June 2012 was given as a first release date, albeit for a smaller wiki. Though a working prototype was successfully released on time in December 2011, attracting (mostly positive) attention from the press, including The Economist, PC World and The Verge, the project ultimately got off to a bad start when an early decision to build their own edit surface ("ES") rather than use the (then rudimentary) ContentEditable ("CE") component built into browsers was reversed in March 2012. As a result, users would have been forgiven if they could not see the difference between the first prototype and the second, released in June 2012 on MediaWiki.org. Indeed, months of development restricted to behind-the-scenes code refinements to both the VisualEditor itself and Parsoid—the new and improved wikitext parsers that underpins it—meant that a year after the first prototype and six months after the original deployment target, the team was still demo'ing almost exactly the same feature set as it had always done: bold/italics, lists, links and headings.

Seven months on and the transformation is complete. The editor now includes template prompting, reference tag integration, image handling and category support—in theory at least. The revised timetable, published a year ago, seems to have been broadly kept to, for better or worse. If the trend continues, anonymous users on the English Wikipedia will get the editor next week, users on other large Wikipedias the week after, and all other Wikipedias where compatibility can be guaranteed a week after that—an adventurous timetable, especially given the storm of comment after the initial deployment on the English Wikipedia and the bug reports it provoked. It is not known when users of Internet Explorer will be able to use the editor, though support for recent versions of the browser is planned.

The contemporary debate

Wikipedia:Wikipedia Signpost/Templates/Analysis Based on an estimated 4 FTEs over two years, back-of-an-envelope calculations suggest the project has cost the Foundation upwards of half a million dollars. It is hardly surprising, then, that the Foundation stands by its assertion that the VE will help reverse the recent decline in editor numbers. Certainly, there can be little doubt that the VisualEditor will, once any bugs are ironed out, make it easier to edit. Whether that translates into a net positive, however, is more dubious. As Ubergizmo suggested back in December 2011, it is not clear that the edits that Wikimedia wikis miss out on every day because they are too hard to edit are of the high-quality variety, drive-by vandalism, or (more likely) a mixture of the two. Other commenters have suggested that new editors may now be additionally bewildered when learning to use talk pages, which still rely on wikitext. In any case, as more data is collected—a trial on new editors started last week—the question of impact should more or less be laid to rest.

In the meantime, the nature of the present deployment—with its focus on existing users—has shifted the focus of discussion onto the merits of providing the VisualEditor to power users by default. Fueled by the fact that only a gadget (instead of a proper preference) allows editors to effectively hide the VE, along with VE's lengthy loading times, bugs (including occasionally adding <nowiki> tags without being prompted), and the lack of reference and template support, many immediate comments on the tool's feedback page have been generally negative. "This is yet another botched new feature deployment by the WMF" wrote one; "turn it off and fire whoever developed it" wrote another. Other comments were more helpful, pointing to the lack of integration with features such as the citation toolbar and special character selection, while yet others posted their appreciation for a tool that "has greatly eased [their] editing"; one user explained that he had " given up on editing" many times, and that VE now allowed him to contribute. The results of an A/B test on new users will be forthcoming. At time of writing, the VisualEditor team say they have no plans to reverse the deployment.

In this issue

3 July 2013

In the media

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

I'd like to see the count that concluded most comments have been negative. I have seen a lot of negative commentary, but also a lot of positive commentary, and would note there is a gadget to hide the VE. Okeyes (WMF) (talk) 16:58, 2 July 2013 (UTC)[reply]

Fueled by the fact that only a gadget (instead of a proper preference) allows editors to effectively hide the VE,'...--ukexpat (talk) 00:14, 5 July 2013 (UTC)[reply]

Indeed; you'll note by the timestamp on my comment here that it was posted before that addition was made to the signpost :). Okeyes (WMF) (talk) 00:38, 5 July 2013 (UTC)[reply]

Apologies—this is why I normally clear talk pages while publishing. We did make changes based on Oliver's comment, as the concerns were valid. Ed ^{[talk] [majestic titan]} 00:46, 5 July 2013 (UTC)[reply]

As noted in this posting, there were at least 200 open (unresolved) bugs when VE went live for all logged-in editors. I hope the Signpost will cover the story, in a subsequent issue, of why the WMF development team felt that it was urgent to provide software with so many bugs to every logged-in editor, rather than delaying the rollout and reducing the number of unresolved issues. -- John Broughton (♫♫) 00:58, 5 July 2013 (UTC)[reply]

As you can see here, Mediawiki has ten times as many bugs open as VisualEditor. Solely the page editing system in Mediawiki has more than 200 open bugs filed against it; if, as someone suggested, an editing system with more than 200 open bugs shouldn't be provided to every logged-in editor, then the old editing window would have to be turned off, too. Perhaps, instead, we can agree that a raw count of bugs isn't the best indication of how well a piece of software is designed and implemented. Whatamidoing (WMF) (talk) 01:45, 5 July 2013 (UTC)[reply]

The linked post excluded "enhancements" and bugs marked as duplicates. Doing that on "Page editing" cuts the number of bug reports down to 100. If you are going to make a comparison of bug counts then one needs to use the same filters. Dragons flight (talk) 23:59, 5 July 2013 (UTC)[reply]

When the VE has a citation toolbar as good as the one in the previous editbox I will give it another try. It is hard enough to try to get new editors to add references. So adding references needs to be as easy as possible. Doc James (talk · contribs · email) (if I write on your page reply on mine) 02:00, 5 July 2013 (UTC)[reply]

Yes, improving the user experience of citations will be a top priority. The system of citations used today is unfortunately very complex -- a combination of the Cite extension, which itself has grown in complexity over time and includes functionality such as references grouping, and the specific citation templates used in some languages/projects (things like {{cite web}} etc.), and the lack of consistency across articles (shortened footnotes vs. full-length footnotes, templates called with <ref> around them vs. templates without it) and you've got a true complexity nightmare. We don't want to hardcode specific templates, so we have to come up with a system that lets communities define the preferred citation templates, if any, and surface those through VisualEditor.

Mapping a discoverable, intuitive user experience against all of this is a significant challenge, to put it mildly. (I'd love to see examples of existing citation UIs that are well-done and offer similar functionality to what wiki markup does!) The current "Insert reference" UI is still very clunky, but it at least lets new users add citations by clicking through a few menus by means of trial and error. I would argue that that's still a better experience in simple cases than having to discover what templates are, what the different parameters mean, etc. through existing markup and hoping you don't forget a pipe or close tag along the way.

Of course I agree the RefToolbar helps a lot, but even it is very opaque. "Templates->cite web", "Named references", "Error check" etc. are not intuitively understandable UI paths. (Not to mention that it's just a tool for inserting refs, not for updating/amending them.) VisualEditor is putting some important bits of infrastructure in place, such as TemplateData (to have a single place that describes all the metadata for a template like "Cite web"), which will help us to create discoverable, intuitive UIs in all languages.--Eloquence * 02:49, 5 July 2013 (UTC)[reply]

And here's a quote from an inexperienced user who had been frustrated by wiki markup before, specifically on the references issues: "I added words that you could click on (i.e. to see that word's wikipedia page). Also, I had no clue how to add references at the very bottom of the article, and put a number at the end of a sentence that you could click on that would take you to that reference at the bottom of the article. Notice how in my edit today I added some examples of medications. You can click on each of them and it will take you to each medication's respective page."

You could point out fairly that the user's edit didn't use a citation template, and I make no claims to merit of the edit -- but having a user who was previously unable to do so successfully add a URL is better than not having them able to add a source at all, no? Now let's turn the citation tool into a really awesome UI and we should really expect to see more edits by new users that add citations.--Eloquence * 03:00, 5 July 2013 (UTC)[reply]

New tagline for the Main Page: "The free encyclopedia that anyone who has high speed bandwidth and who updates their browser version frequently (unless it is one that some developers love to hate) can edit.66.81.249.177 (talk) 18:11, 5 July 2013 (UTC)[reply]

All users can continue to use the source editor, and the additional JavaScript footprint for VisualEditor, when not initialized, is approximately 4KB.--Eloquence * 20:59, 5 July 2013 (UTC)[reply]

Spending half a million dollars on this is definitely a wrong, un-Wiki approach. Wikipedia should be teaching its editors to write extensions like this! Editors have been recruited straight into doing things like template and Lua coding, and elaborate HTML formatting. Wikipedia needs to do more to outreach and encourage editors to make the next steps to start getting into full-fledged Mediawiki development. If we had more editors writing code like this from the beginning then every step on the way would be better-informed by the realistic conditions "on the ground". Wnt (talk) 14:26, 6 July 2013 (UTC)[reply]

Wikipedia seems to want significantly more features than what our volunteer development capacity is. Keep in mind many of the employed programmers were originally Wikipedia editors who made the jump, and then eventually were hired. With that said, if you (or anyone else) wants to learn how to make an extension or hack on mediawiki, come find me on irc (in #mediawiki), and I'll help you learn how. Or at least point you in the right direction (Except not with visual editor because I know nothing about it. Offer does not include teaching you how to program, but there are plenty of tasks for non-programmers to do as well.). Anyhow, I would really love to see more wikipedians (or other folks) taking part in MediaWiki development. So please come join us. We don't bite. Bawolff (talk) 18:51, 6 July 2013 (UTC)[reply]

First I have some remedial learning to do where PHP is concerned, but I may take you up on that at some point. :) Wnt (talk) 17:24, 7 July 2013 (UTC)[reply]

I've got no idea what is so complex about WYSIWYG footnoting capability. My word processor is Apple Pages. You use a dropdown menu INSERT > FOOTNOTE and a footnote number is created and the cursor drops to the footnotes section. You type in the footnote as you wish it to appear and you are done. On WP, if you would want to make a link within a footnote, the logical way would be to highlight the words you want to turn into a link, clink a LINK button which would create a pop up window prompting http://___________________________, type in the URL and click OK and it is done. Why are we reinventing the wheel? Carrite (talk) 19:39, 6 July 2013 (UTC)[reply]

It's those stupid fucking templates that are making this hard, probably. Forget the stupid templates. WYSIWYG is WYSIWYG: the principle is to type things in as you wish them to appear. The Wiki ML code generated should be in the form within [ ref ] and [ /ref ] brackets, which is pretty close to WYSIWYG in appearance. A prejudice against this simple-but-elegant solution in favor of idiotic templates inside curly brackets is probably what is putting you off track. Carrite (talk) 19:41, 6 July 2013 (UTC)[reply]

Yes, the way for new users to add references is any way they please, as long as they get the basic information. There is an advantage in some degree of standardization, but this can be done subsequently by the many people who to attend to such details and have the skill to get them right, assisted by whatever automated devices can be developed that won't mess things up further. Of course, all of this would be easier if there were any agreement within the WP editing community about what form of references was preferable. Since there isn't, and it remains policy that any form of reference is acceptable, the developers of the VE would have done better to implement something just as simple as Carrite suggests; that's what I do when I add references, as I'm more interested in adding references than exactly filling in templates. Their charge was to make it easy to add references, not to match the functionality of someone else's overcomplicated extension. Of course, it takes actual experience in teaching people to add references and in fixing their bad references to know this, along with some understanding of the many unresolved debates about the best way to do it--a few people on the team do have this experience, and they should have more adequately informed the others about the virtues of something simpler. DGG ( talk ) 05:20, 7 July 2013 (UTC)[reply]

What's really important with formatting references is not how the thing is formatted in the editor - it's whether fairly ordinary editors can have access to some cURL/urllib2 like tools to design what are optimally site-specific functions that interpret the source link. Ideally you should be able to paste (or drag and drop!) a link from CNN, and the entire perfectly formatted reference almost instantly appears, because your editor has contacted the source link (or figured out a more concise xml/etc. syndication), figured out it was CNN, and extracted all of the relevant fields, awaiting only your "OK" to be done. Wnt (talk) 07:12, 8 July 2013 (UTC)[reply]

@Wnt: - Parts of such a system now exists - see Help:Citation tools. Editors (like me) would have been falling over themselves to use VE if it could automagically covert a url to a full citation; and consider the value if VE also estimated the likelihood that a source was reliable (obviously only a rough guess, but still - what a help in evaluating sources posted by others). But no, what we got was a kludge so that the VE team could meet an (arbitrary) deadline for providing a "beta" version to all editors. -- John Broughton (♫♫) 04:47, 9 July 2013 (UTC)[reply]

Hmmmm... do any of those tools return JSONP format that can be included into a user script? (I think - any scripting in Wikipedia tends to run into unexpected security restrictions) If so, or someone makes some, it might be more directly integrated into the Wikipedia edit window... Wnt (talk) 05:59, 9 July 2013 (UTC)[reply]

Ralph Fiennes has something to say about this :) --NaBUru38 (talk) 14:50, 10 July 2013 (UTC)[reply]

The Signpost is written by editors like you — join in!

Home

About