Wikipedia:The Wikipedia Adventure/Story
Wikipedia is not a game, but learning it should still be fun.
- 1 Motivation
- 2 Introduction
- 3 Impact
- 3.1 Phase 1 pilot and beta metrics
- 3.2 Overall satisfaction
- 3.3 Quote highlights
- 3.4 Quantitative analysis
- 3.5 Qualitative analysis
- 3.6 In their own words
- 4 Participants
Problem: Wikipedia is a very difficult environment for new users. In addition to a new interface, the community has its own policies and customs; and that is on top of the already challenging task of approaching the world of information with critical thinking and neutrality.
Reality: Very few people edit Wikipedia, far fewer than those who read it or who have the capability or interest to edit it. Many struggle for days or weeks to learn the basics. Most find the environment off-putting and intimidating.
Solution: Create an educational, interactive, web-game using a simulated Wikipedia interface which leads new users through a series of realistic missions to familiarize them with the mechanics, navigation, philosophy, and practices of actual editing.
Goal: Have new users leave the game feeling capable to start editing, knowing what to do if they run into obstacles, and feeling some sense of success, pride, and possibility as new editors.
|The Wikipedia Adventure|
The Wikipedia Adventure is a 7-mission interactive guided tour which introduces new editors to basic editing, social, and policy skills. It was started in 2011 as a script and built in 2013 as an Individual Engagement Grant through WMF.
Learning to edit Wikipedia could be hard, frustrating, confusing, and overwhelming for some editors. The purpose of this game is to remove all of that and create a learning experience that 'curates' a person's first impressions and lessons of Wikipedia. So, rather than walk away from the site feeling like it is incomprehensible or (worse) threatening, users will graduate from The Wikipedia Adventure ready to face the challenges and opportunities of the real site and community.
One of the Wikimedia Foundation's core goals has been new editor engagement and active editor retention. The Wikipedia Adventure targets these two goals by reducing the steepness of the learning curve to joining the community. If more new editors have a better first experience with Wikipedia's concepts and mechanics, more will go on to be active editors. If those editors are prepared to deal with some of the common pitfalls of the community, then they will be more likely to remain on as productive contributors to the community.
The Wikipedia Adventure is an online guided journey, a learning tutorial, and an educational game designed to make an editor's first 100 edits a positive and encouraging experience. The game takes the user from the first rudiments of editing—registering an account, creating a user page, basic markup — all the way through to more intermediate tasks such as adding images and references and asking for help at noticeboards.
The game centers around the hypothetical article Earth (actually a modified form of the Simple English Wikipedia's article on Earth). Earth was selected as a focal point, because it has literally universal appeal and avoids the cultural favoritism involved in choosing a representative article that may only appeal to some readers (e.g. The Beatles or Mother Teresa).
Through a realistic but not 'live' emulated interface, the user gets to experience what happens at a real article. It begins with an invitation from another user to work on the Earth article and involves interactive tasks such as typo-fixing, identifying reliable sources, crafting writing from a neutral point of view, understanding core content policies, and even dealing with vandalism and nonconstructive edits.
The game focuses on more than just the technical mechanics of editing, infusing the script with mock interactions with simulated 'real' editors. Some of them are friendly, some are less so—and the user has the experience of learning about how to communicate in a productive and effective manner. At the heart of the game is the lesson of cooperation and collaboration, that is what makes Wikipedia work the spirit of the people who write it and that anyone can be a part of it if they learn a few basics.
Throughout each mission editors complete editing and policy challenges about Wikipedia, so that their progress in the game is matched by their progress as real editors. The ambitious goal is that new editors would not feel out of place or ignorant as they confront the actual editing environment of Wikipedia but instead would be prepared and even enthusiastic to get started.
As a result of this game, Wikipedia will have another tool in its arsenal to educate new users and improve their initial experience as members of the community. The game serves the many, but in a personalized way. As an online game, it scales easily and can be delivered to tens, hundreds, thousands, or hundreds of thousands of people. There is no limit to the game's potential exposure; if effective, it could be a standard element of welcoming new editors across all of our 280+ languages.
It's important that the game's expansion would continue beyond the initial implementation. This would be facilitated in several ways. For one, the game's script and code are all already released under open source, 'free' licenses that permit use, adaptation, or even commercial applications. That would be essential to allowing others to build on the platform. Second, the platform itself would be designed to allow other users to build their own tutorials and modules. We hope that the editor will propose or build additional missions, and translate the game into the context or language that suits their community and niche best.
In October 2013 we ran an extended bugfixing alpha-test with over 50 editors and 200 bugs identified and fixed. In November/December 2013 we ran a beta-test in which nearly 10,000 editors were invited to play and 600 did. In January we crunched the data from that test. Here's what we found:
Phase 1 pilot and beta metrics
- TWA players made more edits: New editors who played TWA made 1.2x more edits than a control group of similar but non-invited new editors. Players made 1.9x more edits than those who were invited but did not play the game.
- TWA players were more likely to make 20+ edits: TWA players were more likely (1.2-1.7x) to make 20+ edits than either control group. TWA players were also more likely to make 0 edits than the control groups, however.
- Players who finished the game made the most edits: Players who completed the game made 3.2x more edits than those who only started the first level of the game, and were 2.9x more likely to make 20+ edits.
- Players enjoyed the experience and felt more confident: 87% of players surveyed were satisfied or very satisfied overall with the game. 89% said: 'TWA made me more confident as an editor.' One TWA player said: "It really left me feeling prepared to make future edits." 89% also said, 'Lots of new editors should be invited to play TWA.'
A possible explanation for these findings: New editors who play TWA appear to be making their test edits within the game rather than on articles. Those who continue to edit Wikipedia after playing demonstrate the confidence of a more highly active editor.
- 87% were satisfied or very satisfied overall
- 89% said 'TWA made me more confident as an editor',
- 89% said 'TWA helped me understand Wikipedia better'
- 77% said 'TWA made me want to edit more', 6% disagreed
- 79% said 'TWA made me feel welcomed and supported'
- 71% said, 'TWA helped me know what to do next', 9% disagreed
- 80% said, 'TWA prepared me to be a successful contributor to Wikipedia'
- 75% said, 'I enjoyed playing it', 6% disagreed
- 89% said, 'The game is a good way to introduce new editors to Wikipedia'
- 89% said, 'Lots of new editors should be invited to play The Wikipedia Adventure'
- "I enjoyed the idea of editing a fake article for practice - in fact, when I first saw the game, I immediately hoped it would incorporate some sort of actual editing rather than just theory or questions or something."
- "Well, what's there not to like, or to have an opinion on...the game is great, most-of-all for us users that are just starting up in Wikipedia."
- "I didn't know there was talk and discussion among users until I played the game...I just thought you could make comments and report on individual pages."
- "I've seen and heard companies, including my own, talk about learning through 'gamification'. I found TWA to be the best example of gamification I have witnessed to date."
- "TWA was very informative and helped pull back the curtain on some of the fundamentals of editing."
- "I think TWA at the moment is a great stepping stone for new users such as myself. I would love to see it expand to include more 'advanced' topics that can be optionally covered by the user."
Extended Quantitative and Qualitative analysis
We analyzed 3 groups of 165 editors each in November/December 2013. All were selected from Snuggle's desirability algorithm as likely good-faith contributors. 10,000 editors were invited to play using a mass talk page message invitation.
All groups made 1 edit before they were sampled, and made at least 1 edit afterwards, to ensure that they didn't just wander away. In other words, groups 2 actually saw the invites, and group 3 made an edit after the game as well. We found that:
A note on the analysis: we ran this data after a very brief amount of time--barely 4 weeks. A fuller timeframe for analysis which we will conduct by Spring 2014 will be more robust and have more meaningful signals in them. (We plan to conduct statistical significance tests and reports about editor retention and edit persistence).
Total average edits
Article space edits
Talk page edits
Number of articles
We surveyed the 600 editors who at least made it to the first stage of mission 1. We sent these editors talk page invitations to a Qualtrics survey using EdwardsBot. 42 editors responded between December 23rd and January 4th.
The hypothesis we set out to test was that play could be thoughtful and fun could yield meaningful experience and education. The survey data supports this conclusion.
We also aimed for a target demographic of college-aged men and women. The most common given age group for appropriateness was that demographic, so it looks like we aimed right. It's also worth noting that the bell curve was fairly 'thick' around this demographic, and survey respondents thoughts TWA would be appropriate for many age ranges, especially those 13-29 (but also younger those than 13 and 55+).
Possibilities for expansion
In their own words
What they liked
What they didn't like
I'd like to note that the most negative feedback consistently came from one respondent who had 100,000+ edits. While I do not discount their points--echoed by earlier design debates about the game's playful or even youthful nature--it needs repeating that the target for the game is new editors, and these contributors are different and have different needs than experienced contributors.
What they wanted more of
I can't help but note that this winter, Jackson Peebles' reimagining Wikipedia Mentorship grant proposal would likely have been funded, were it not for his passing. This is an incredibly fruitful area for future improvement and we should not pass up the opportunity to reinvent it.
- More details: Phase 1 report