WMF's latest strategy document shows successes, vagueness, and the need for better data
State of the Wikimedia Foundation
The Wikimedia Foundation this week released a State of the WMF report, a 53-page "snapshot" of where it is and where it wants to go in the future. The Signpost was provided an advance copy of the report, which senior communications manager Juliet Barbara told us "is primarily a retrospective document, with some foreshadowing for only the 2015 calendar year ... the primary audience was internal, but we planned to make it public from the very beginning". By contrast, the 2015–16 annual plan, in preparation, will be a detailed look at the WMF's goals, budget, and staffing plan. State of the WMF, the subject of a just-released Wikimedia blog, is billed as part of a strategic process—"not as a set of goals or objectives," it says, "but rather as a direction that will guide the decisions for the organization".
This less tangible goal is reflected in the text. It shares much with the tenor of corporate strategic statements, which have evolved to favour positive, general information over specific commentary and self-analysis. This is not to say that the 53 pages lack specifics, but rather that the message is often expressed in relatively vague terms (Product will be "working with community champions in various projects"), compared with its promise to offer "data-based results, project impact, challenges, and how our work ties back to our mission." Rather than "a transparent and candid reflection on our accomplishments, opportunities, and challenges", parts of the document are surprisingly evasive when it comes to detail, and in many places hard for outsiders to test against the facts. While there was explicit information, there were also coded messages ("a shift in focus from money and process to impact and non-monetary support").
This was all to be expected in a document that is an assemblage of submissions by each of the WMF's eight departments. Whether intended or not, this has resulted in a mild sense of defensive competition among departments to present their best face rather than exposing themselves at an early stage by announcing solid initiatives in what appears to be a pre-annual-report think-tank.
Whether explicit or implied, the theme of inadequate data is lurking in a number of places throughout the document, to the extent that in some respects the organization seems hamstrung by it ("An overwhelming percentage of project knowledge is not available as structured data"; "The Wikipedia Zero team needs to understanding [sic] potential alternative metrics"). Barbara responded to this proposition: "the need to collect and use data to make decisions is a major theme of the report [and] we are formalizing that commitment as an organization." We asked the Foundation's chief communications officer, Katherine Maher, what steps the WMF will be taking to resource and guide its data acquisition, given the apparent importance in the document of better baseline and ongoing data:
||Our new COO Terry Gilbey will play a major role in helping us identify and adhere to metrics-based decisionmaking. This week, Terry announced over the next few weeks the WMF will begin a structured implementation process for the Call to Action 2015 objectives, including the commitment to data-based decisionmaking. This will include defining clear goals and success criteria for each initiative, a C-level sponsor for each initiative, and team leads to help manage the process.
Engineering and product
The report is the first strategic statement by the Product and Engineering team in its restructured form—originally announced in November 2012. The new arrangement, a refocusing as two separate departments, is "to ensure development of best practices", according to the new executive director Lila Tretikov. The former head of both functions, Erik Moeller, continues his work as vice president of the product and strategy side, and deputy director of the WMF, while Damon Sicore now leads the engineering side. Based on this shift in responsibilities, the State of the WMF 2015 provides two separate reports on these functions.
Average edit-save time in milliseconds, late 2014
The Engineering Community team co-organized several hackathons and tech talks, and participated in outreach programs such as FOSS Outreach Program for Women and Google Summer of Code. The team coordinated the replacement of the bug management tool Bugzilla with the software development platform Phabricator. The entire engineering department is now supported in fulfilling their commitments by a new team practices group. While the Foundation successfully responded to outside security threats such as the Heartbleed bug and the POODLE attack, the Signpost considers the security engineering department to be understaffed. However, a new engineer is currently being hired to address security issues.
The product team focused on the mobile challenge by releasing rewritten apps, with several new engagement features, such as in-app editing, offline reading, collapsible infoboxes, contextually relevant images, and a short description of the topic at the top of the articles taken from the corresponding Wikidata item, and a "read more" feature. Between Apple and Android, the Wikipedia app boasts a six-million-strong active userbase and an average rating of 4.0–4.5 out of 5 by users. A new mobile Web interface was launched that makes Wikipedia easier to read and edit on a mid-sized screen with the help of a new mobile version of VisualEditor. The Signpost notes that the upload of images to Commons through mobile web upload caused massive spam of non-free and abusive images that got deleted. The desktop version of the Visual Editor was enhanced with table editing and adding citations and reached more than 5 million uses in 160 Wikimedia wikis; it will be rolled-out to all major projects and platforms in 2015. The analytics team has provided critical data and systems for measurement. Some other tools have been developed: the communication tool Flow that allows contributors to watch single sections on talk pages has been enabled on some Wikipedia project pages on the French and Catalan Wikipedias, and will replace LiquidThreads pages, the new contribution stream WikiGrok, and the Media Viewer.
A caricature of the superprotect issues
Because the Media Viewer was deployed with too many bugs, this Flickr-like multimedia browser caused negative reactions from many contributors. In a lessons learned section, the WMF reflects on successes and mistakes in developing the browser, including the lack of tools to get productive feedback from the community, and the absence of a clear metric for success. The report does not mention the superprotect issue, which resulted in a storm of heated discussions in the communities, including a letter that was signed by about 1,000 contributors, and media coverage, particularly in German outlets. The Foundation admits that such "expensive and slow" changes impacting desktop users led to an aversion to working on desktop issues and a shift to mobile development; but both areas, the document concedes, need further improvement.
On the mobile side, the WMF intends to create new ways to participate in Wikimedia projects, given that MediaWiki does not display edits and authors outside of the article histories—will there be something like WikiTrust or the Article Monitor? Concern is expressed on the desktop side that external projects like WikiWand would oust the Wikimedia Foundation in its function as operator when design changes "clash with edge cases or with editor-focused functionality":
||We risk being left behind when it comes to offering the best experience reading Wikipedia.
Just how this will affect the editing community is unclear, but Product and Engineering is planning to strengthen community input into product priorities and develop update and feedback channels. Still, one consequence could be the disabling of admin changes in favor of a code-review process in the MediaWiki namespace, as proposed in Phabricator.
The document reveals concern about the low availability of structured data in the Wikiverse and therefore a lack of consistency in styling. Neither its support of Wikidata, mainly developed by Wikimedia Germany (e.g. with WikiDataQuery), nor their joint plans on Structured Wikimedia Commons are mentioned. Neither is one of the WMF's most invasive actions, the current Single-user login finalization. While on the one hand, unifying all accounts is the basis for long-anticipated improvements like global watchlists, global notifications, and efficient Flow implementation, on the other hand it requires forcible rename actions to resolve naming-clashes. D
The extent of Wikipedia Zero as of the beginning of March.
Wikipedia Zero may be in trouble. A major initiative designed to deliver Wikipedia gratis to millions of people (currently about 400 million) in the developing world, the "State of the Wiki 2015" document revealed that the program is facing "major challenges" in its implementation. The document reveals disconnects between the Wikipedia Zero team and the WMF as a whole. For example, the team accuses the Foundation of "not dedicat[ing] resources" to its target audience in the Global South, and believes that adoption rates have been less than expected because the WMF does not advertise the product. This problem, which results from the trademark policy and the Wikimedia community's long-standing opposition to advertising, is exacerbated by a lack of organic growth—a matter that Katherine Maher, the Foundation's chief communications officer, told us could come up for discussion. Furthermore, between phone carriers and Wikipedia Zero, zero-rating programs are becoming less popular to carry. Perhaps worse:
||The biggest concern for Wikipedia Zero is that we do not yet see evidence that it is reaching the target audience—the world's poorest people who cannot afford mobile data charges—at scale. We still do not see organic growth in usage. And our own data on pageviews by language version show roughly 90% usage in English throughout South Asia, indicating the program is actually reaching more privileged segments of society. … Making Wikipedia free of data charges is not driving usage in underserved segments.
These come on top of months of criticism about Wikipedia Zero with respect to net neutrality. The WMF's former talent and culture officer Gayle Karen Young told the Washington Post that the WMF has a "complicated" relationship with the ideal: "Partnering with telecom companies in the near term ... blurs the net neutrality line in those areas. It fulfills our overall mission, though, which is providing free knowledge." The WMF has defended itself by pointing out that they do not pay carriers, nor offer it as part of a bundle; but carriers are already beginning to chafe under these restrictions. Net neutrality activists take issue because, as the Washington Post wrote: "This preferential treatment for Wikipedia's site helps those who can't afford to pay for pricey data—but it sets the precedent for deals that cut against the net neutrality principle of non-preferential treatment for any website."
Related to the net neutrality concern is the recent Wifione arbitration case. Wifione, a long-time Wikipedia administrator, was banned from the site this February after a long-term campaign to enhance the reputation of the Indian Institute of Planning and Management, a controversial for-profit business school. Newsweek credited Wikipedia as being a major factor in the school's recent prosperity: "For poor students from rural areas of India, Wikipedia was often the only source of information they could use to research which business school to attend. ... In 2013, IIPM got an unexpected boost for its page [from] a new initiative launched by Jimmy Wales's Wikimedia Foundation ... The program, Wikipedia Zero, launched in India and other parts of the developing world." E
The most important news is that the Foundation met its fundraising goal for the 2014–15 financial year six months ahead of schedule. The year-end total will considerably exceed fundraising goals this year, continuing the Foundation's marked upward trend in revenue. In 2013–14, "the Fundraising team raised $52.6 million, exceeding the annual plan by 5%". For the current year ending June 30, Fundraising aimed to generate $58.5 million. That total was reached on January 7, six months ahead of schedule. $32.3 million was raised in the December 2014 campaign alone. The "mobile full screen banner increased donations by 250% over the 2013 banner format." Donations from the December email campaign were up 20%.
The Foundation "intentionally exceeded [its 2014–15 goal] to prepare for future challenges", suggesting that internal goals differed from those publicized. What are these future challenges? Banner-driven revenue is tied to pageviews. The WMF is concerned that "all traffic is trending down in the countries that have traditionally been our core financial contributors. If this trend continues, our model will be challenged in 2015."
In the US, total pageviews dropped 8.6%; in Belgium and the Netherlands they dropped by 30% and 31%, respectively. Traffic is up in countries like India (+13%) and Iran (+168%), but this is not where significant funds are raised ("90% of funds come from North America and Europe"). These drops in pageviews in the US and Europe are not shocking to the WMF: "it's consistent with general mobile engagement trends ... people tend to have shorter sessions on mobile devices and often use these devices to find quick facts on the go, whether using Wikipedia or other services. Having this baseline data is important for our product and mobile teams. As traffic continues to shift from desktop to mobile, this data helps our teams better understand reader and editor behavior on mobile devices."
The decline in traffic "may be linked to readers accessing Wikipedia content off-site, for example via Google's Knowledge Graph or Facebook". In addition, the shift from desktop to mobile, including in the US and Europe, means that on average banners are less productive: "According to our projections, our revenue from our year-end English fundraiser would have decreased by 43% had we run the same campaign as last year ... We need more effective messages to produce the same result."
To meet future challenges, the Foundation aims to grow "income channels, including mobile, email, foundations and major gifts" and to develop "new channels for reaching existing and potential donors." Continuous year-round research, testing and feedback will continue, and the next six months will see email campaigns in Hebrew, Swedish, Hungarian, Norwegian, Spanish, Catalan, Chinese, Portuguese, Brazilian Portuguese, and Ukrainian. The vice president of partnerships will "develop and oversee strategic collaborations with key partners in the free knowledge movement." AK
Talent and Culture
2014 was, in words of the report, a "year of change" at the Foundation. The Talent and Culture department, tasked with maintaining this change in staffing terms, is a fanciful name for what would otherwise be called "human resources (HR)". The Talent and Culture team is in a transitional phase at the moment: this month saw the departure of its head, CTCO Gayle Young, and the on-boarding of COO Terence Gilbey, now serving as its temporary CTCO. Just as the broad strokes of the talent and culture team's reporting allude to transition, the team itself is in such a state.
There are several references to designing and pursuing a "culture shift", defined to be the use of "process levers" to "work to build desired culture attributes more robustly". What these levers and attributes are is left unclear; this year's strategic report will likely reveal more details about what this means, as well as about an allusion, not explicitly defined, to the need to construct a rung of "second-level leadership" in the WMF.
When Sue Gardner was hired in 2007 she was given the rare opportunity to grow into her role: when she arrived the Foundation spent just $3.5 million per year; by the time she announced her intent to leave it the organization the annual fundraiser was bringing in almost that much money per day. The Foundation had grown in that time from a ten-person staff and a fistful of servers to a multimillion dollar public-facing non-profit with 160 employees, 100 of whom arrived in just the two immediately preceding years. The growth has not stopped: today the WMF has a $58m budget and a staff of more than 280. Gardner was faced with the organizational challenges of building a robust outwards-facing organization and expanding the staff; Tretikov faces a different challenge—to make it run better, using her very different background as a tech executive with five patents and a skillset in understanding and managing technology and engineering. The growing pains that come with such a pivot are the likely underpinnings of the "culture shift". R
The most interesting aspect of the communications team's reporting could be regarded as a runaway success: last year's relaunching of the WM Blog (formerly the WMF blog). The blog published 306 posts and reached 84,589 monthly unique visitors, allowing and encouraging the community contributions to site content as "a global storytelling tool for the communities" where previously it had served solely as a route for corporate communications from the Foundation itself.
At the same time Communications is aware of a need to improve its own core reporting capacity. A thread recycled in other parts of the document by other departments (grantmaking in particular) is the need for the communications team to, in its own words, grow "core team capabilities to support more strategic and far-reaching communications activities"; in summarizing its activities for the year 2014 the communications department states that "the team worked to shift its focus from reactive to proactive communications ... being told broadly, accurately, and with compelling messages." R
Legal and Community Advocacy
More publicly visible, LCA administered the contributors' defense fund at least twice—the report highlights an incident involving an administrator on the Greek Wikipedia—to defend contributors from legal action stemming from legitimate editing of the encyclopedia. LCA was also responsible for streamlining some internal WMF communications processes aimed at improving efficiency.
Global censorship is on the rise especially in countries with large populations of Wikimedians including Russia and even some states in the European Union. There is also growing support for "copyright reform".
||While copyright reform presents an opportunity for the movement, it also represents potential risks, as various interest groups around the world will press for SOPA-like changes and term extensions, both directly through copyright law and indirectly, through mechanisms like trade agreements.
In early 2015, community advocacy was consolidated into the new Community Engagement department, headed by former WMF deputy counsel Luis Villa.
Despite the purported successes, there are misgivings. One appears to be the inability or unwillingness to take on the child protection function from the English Wikipedia's Arbitration Committee, the subject of repeated requests to the WMF. This raises questions as to whether Community Engagement should prioritize putting child protection in the hands of trained, paid professionals rather than volunteers with no formal training who were elected by the community to perform an entirely different role. GP
Wikipedia's identity crisis
William Beutler, author of the blog The Wikipedian, commented about the use of the phrase "identity crisis", which appears no less than three times in the report. In the Communications section: "The identities of Wikimedia, Wikipedia, other projects, and the Wikimedia Foundation are unclear from a branding perspective." He writes: "Concern in the document about the naming of Twitter accounts and Facebook pages speaks to a larger dilemma. Wikipedia's massive success has made it a household word, but the more inclusive term 'Wikimedia' has never enjoyed the same level of recognition. As a movement that wishes to encourage outsiders to join its project, Wikipedia (Wikimedia?) needs to pay close attention to how it communicates about itself.
"Different solutions have emerged: in GLAM, the term "Wikipedian-in-residence" is used, even as the WiR is likely to be interacting with Commons or other Wikimedia projects. The Wiki Education Foundation, by the very act of naming itself, acknowledged the lamentable fact that a lot of people simply call Wikipedia by the term "Wiki". And so do many Wikipedia editors, although most are careful enough to lowercase it, e.g. "wiki editors".
"Obviously, the proliferation of terminology owes a great deal to the complex nature of the Wikimedia movement, and can be a source of confusion within as much as without. What to do about it? Two potential solutions emerge, long and short.
"Just as the Wikimedia Foundation maintains a well-developed set of visual identity guidelines, it should eventually spearhead the creation of a stylebook to straighten things out. When in doubt, when in public, say "Wikipedia", and reclaim the twenty seconds you would have spent explaining the difference to your puzzled conversation partner." WWB
Wikipedia's conceptual barrier
The Fundraising section stated that "mobile traffic increased by more than 60%, while desktop traffic remained flat or declined [in 2014] ... The overall decline in traffic may be linked to readers accessing Wikipedia content off-site," like Google's Knowledge Graph or Facebook. Engineering and Product discussed the challenge faced by software like WikiWand: "Third-party reader interfaces develop more quickly than Wikipedia's reader experience can change."
Lately, it seems that an increasing portion of Wikipedia's readership is viewing the encyclopedia without actually accessing the encyclopedia or fully participating in the Wikipedia experience. This is not limited to mobile traffic or alternative software; it happens with the main driver of traffic to the encyclopedia, Google. For example, in a 2012 study of undergraduate use of Wikipedia in the Journal of Academic Librarianship:
||... most of the students interviewed expressed that they usually reach the Wikipedia site by searching for a term on Google or another search engine and not by going directly to Wikipedia ... In fact, several respondents said that they 'never' begin a search that concludes with a Wikipedia entry by first visiting the Wikipedia.org website. (Colón-Aguirre, Mónica; Fleming-May, Rachel (2012). "'You Just Type in What You Are Looking For': Undergraduates' Use of Library Resources vs. Wikipedia". The Journal of Academic Librarianship. 38 (6): 391–99. )
While these particular users are visiting the encyclopedia itself, they are not using it as an encyclopedia—just as one of many pages that appear in a Google search.
The common denominator is that these users are readers, not editors. The second group will always be a smaller subset of the first, but these technological changes and usage trends threaten to widen the gulf between them. Wikipedia Zero may expand this gulf exponentially, turning large untapped parts of the world into a mass of passive Wikipedia readers without any real ability to edit the encyclopedia in a serious way. This gulf threatens more than just fundraising and traffic statistics. The wider the gulf, the more difficult it is to recruit and retain editors and to maintain the quality of the encyclopedia.
For all the branding of Wikipedia as "the encyclopedia that anyone can edit", most people think of Wikipedia as something someone else edits and never think of seriously contributing themselves, much like people don't give any thought to how all that movie data gets into IMDB or who puts all those cat videos and memes in their Facebook feed. For them, it is utility that pumps information into their life from Internet Co.™ as long as the bills are paid, not a community garden that they should help tend.
In terms of technology and access, the real cause of this gulf is conceptual. Many people do not think of encyclopedia editing as something they can or want to do, or lack the necessary self-confidence to think they might contribute to an encyclopedia—a kind of self-doubt that is absent in ideologues of all stripes, which is why they appear so frequently on Wikipedia. If anyone's equipped to deal with issues of information creation, access, and dissemination, it's librarians. But librarians with many years of professional experience can be completely befuddled by simple Wikipedia concepts or freeze up before hitting the submit button to make a minor edit. If they are not able to navigate this conceptual boundary, then we're definitely not doing enough to make it possible for potential editors in general. By their very nature, vandals and hoaxers cross boundaries with ease; the Wikimedia community have to figure out how to get everyone else to traverse this conceptual boundary as well.
Wikipedia was created by people who believed they could change the world through technology—and they have. But now that Wikipedia is an institution, the Foundation has to move beyond technical and into conceptual change if they are going to meet the challenges of the future, and insure that editors will continue to join us in maintaining this institution. G
- Responses and critical commentary are invited in the comments.
Keep up with The Signpost