Wikipedia:Help Project/Usability Testing 2

A series of usability tests were carried out by the wub in Sep/Oct 2012, as part of the help pages community fellowship. These were focused on the existing design and the proposed redesign of Help:Contents. These tests were carried out remotely through UserTesting.com. This page contains a summary of the results, along with links to view the test recordings.

For the previous in-person tests see Wikipedia:Help Project/Usability Testing 1.

Test method

Each user was given a scenario e.g. "You need help finding out how to cite Wikipedia in a class project."

They were then sent to Help:Contents (the current design) and given the following prompts:

What do you like about this page? What do you not like about this page?
Take a moment to read through this page, and then tell me which of of these links you think will lead to the information you need. If you think that the information could be under more than one of these links, please say which ones you think are most likely to lead you to the answer. Before you click on any links, please briefly explain why you choose that link.
Click on the link. Is this page what you expected it to be? Why or why not?
Does this page contain the information you were looking for? If so, please identify the helpful information on the page.
What, if any, information that you expected to find on this page is absent?
If this page didn't contain what you were looking for, please click "back" on your browser and try another link that looks like it might help you find what you’re looking for.

Next they were sent to Help:Contents/B (the proposed redesign) and given the same prompts.

Finally a number of follow-up questions were asked.

Citing Wikipedia

Scenario: "You need help finding out how to cite Wikipedia in a class project." Test video

User info

Age: 41
Gender: Male
Experience: Average Web User
Daily Web Usage: 3-5 hours

Current page

The user thought the page had "a lot of good information" and was also interested in the tip of the day. On the negative side he commented that the page "does throw a lot of information at you at the beginning", and has "too much vertical scroll". (Note that a Wiki Loves Monument banner was running, which increased amount of scrolling on both pages) Given the scenario he decided to click on Links and References to find the information. He then found Wikipedia:Citation tools under the References heading. He believed this met the goal of the scenario, although it actually covers adding citations to Wikipedia which is not what we were looking for. In fact how to cite Wikipedia is mentioned in Researching with Wikipedia which can be found via Help:Contents/Browsing Wikipedia.

Found useful information for scenario: No

New page

Positive: "I like this page because it actually cites examples, these are things I might want to do." Negative: "There's still a lot of dead space at the top" By elimination he concluded that the scenario fell under "I have another question about Wikipedia", and went to the Frequently Asked Questions. "How do I cite Wikipedia" is the 5th question on that page, which he found successfully and was directed to Wikipedia:Citing Wikipedia which had the desired information.

Found useful information for scenario: Yes

What frustrated you most about these pages? Why?: Very complex information, but a good start to learn what I need to do to cite Wikipedia. It reads like a reference manual, which relies on the user to do what's necessary to learn it., and that's expected.

What did you like about the pages? Why?: Plethora of info, was inundated with what I believe to be good reference material and though I many need some help from a peer, at least being able to speak intelligently on the subject.

Did you prefer the first or second page? Why?: Second page seemed more inviting to me. Though it could be improved even more to become a tutorial, as a daunted user it made me feel more comfortable.

Do you have any other feedback?: No, I do not. I have not used Wikipedia for anything but reference and learning, and visit it nearly daily.

Article was deleted

Scenario: "An article you created on WIkipedia just got deleted because someone said it was not notable, and you want to get an answer from an actual person about how to fix this."

Test video

User info

Age: 49
Gender: Male
Experience: Average Web User
Daily Web Usage: 1-3 hours

Current page

The user was initially a little confused by the page not looking like a normal Wikipedia article page, and pointed out that the blue colouring and formatting reminded him of a Google ad. His initial thought for the scenario was to click on "Asking questions" in the main body of the page, since it had the description "Contact another user". After looking round a bit more he spotted the "Ask questions" link in the header bar which actually leads to a different place. He noticed the different link targets and commented on how confusing it was, deciding to go with "Ask questions" as he believed it would allow him to contact "Wikipedia itself". He liked Wikipedia:Questions: "It has direct links to exactly where I expect to go, and it has explanations of where these links are going to take me.", and determined that the Help Desk would be the best place to ask.

Found useful information for scenario: Yes

New page

"This is much less busy than the landing page I've seen previously" "This tells me that this is a help page" "The icons immediately are catching my attention. They're basically breaking up the information so I'm not confused by it like on the previous page" "Nothing that I really don't like about it. It's clear and it's to the point."

He goes via the "report a problem with an article" heading, to Wikipedia:Contact us/Article problem. Not the typical use for this heading, but it seems to serve well. He did point out that it would be nice if the heading itself was a link, but this isn't really possible as some sections require multiple links. Was happy with this page too, scanned the list not seeing his specific issue but quickly found the Help Desk link at the bottom.

Found useful information for scenario: Yes

What frustrated you most about these pages? Why?: The first landing page was confusing because I wasn't sure about the navigation. The links seemed like ad placements.
What did you like about the pages? Why?: I liked that each page was instantly recognizable as a Wikipedia page and I understood immediately what I could do there.
Did you prefer the first or second page? Why?: Second. I much preferred the layout. The use of icons to bullet point the information was much more intuitive.
Do you have any other feedback?: Wikipedia is a fantastic resource. Keep up the great work.

Outdated info

Scenario: "You notice that an article about your favourite sports star has outdated information, but you’re not sure how to fix it yourself."

Test video

User info

Age: 45
Gender: Male
Experience: Advanced Web User
Daily Web Usage: 1-3 hours

Old page

"Don't like the blue" "I use it [Wikipedia] probably on a weekly basis, but at the same time it's never a pleasure to look at is it?" "It's useful" "Aesthetically, visually, it's not nice."

At this point it became clear the user did not understand the test scenario, that he should be looking for help on the page he was sent to, and was expecting to see the article on the sports star itself. Therefore he skipped through the remainder of these tasks.

Found useful information for scenario: n/a

New page

On going through to the new page, the user got somewhat back on track. "I like it because it's got big text. I like it because it seems to be in sections. I would like a border round it, but I do like the fact it's actually got headings etc." "Like the fact it's highlighted, it's bolder" (referring to bolding in headings).

Decides it would be best to click on Getting started under "I want to learn more about editing Wikipedia".

"Is this page what you expected it to be? Yeah, it's got a Tutorial, basically."

"The whole site is link-crazy, and I think that's its strength - it's links, everyone knows that. But at the same time there's no need for your tutorial pages and help pages to resemble the rest of Wikipedia." "Intuitive navigation doesn't work by just putting links everywhere"

Took a quick look at the tutorial and thought the content seemed quite good, although not the visual appearance.

Found useful information for scenario: Yes

What frustrated you most about these pages? Why?: The test was the most frustrating part, some thought really needs to be put into the scenario and questions so they make sense to the tester. As for the pages themselves their was nothing to distinguish them apart from any other page of wikipedia, surely some separate coding could have been written for the help pages. And also intuitive navigation does not consist of asking yourselves a question then putting the answer somewhere on that page this is not correct way to communicate with the customer
What did you like about the pages? Why?: it's functional, it's wiki.....
Did you prefer the first or second page? Why?: second, at least some attempt had been made to use headings and bold fonts, if we start heading down that road we will soon be into borders, backgrounds and resembling something appealing to read and look at.
Do you have any other feedback?: Yeah sorry to be so negative but sites get tested for genuine feedback, and if nothing else the feedback was genuine, I believe the editing and contributing is something people do for free, give them something good to work with at least

Image caption (1)

Scenario: "You want to figure out how to upload a picture to a Wikipedia article and you can’t figure out how to add a caption to the picture"

User info

Age: 60
Gender: Female
Experience: Average Web User
Daily Web Usage: More than 7 hours

Test video

Current page

Confused by the namespace checkboxes for the search box. Referring to main blue box: "I like that it's got all these categories" Decided to click "Files" link for the scenario. "It's not exactly what I expected to see but it's pretty good. I would have expected to see the help broken down into more categories. Maybe less on 'Other' and I would have expected to see a bigger 'Help' section." Thought about clicking "Guide to captions" next but then dismissed it as it was under the Policy and guidelines header. Instead went to "How to insert and use pictures in Wikipedia articles". Couldn't find captions anywhere in the table of contents, so clicked back.

Next went to "How to upload files". Still couldn't find it, so went back to Help:Contents and typed "how to add caption" in the search box, unticking all the checkboxes other than 'Help'. Somehow skipped over the link to Help:Visual file markup (terrible pagename!) and went to Help:Table#Captions! "Well that's the HTML... I don't know where I am. Very confusing."

Returned to Help:Contents once again and tried Editing Wikipedia link. Didn't see anything there. "I guess the best thing I found was with the search engine. I'm not having very much luck."

Found useful information for scenario: No

New page

"I like this page better than the last help page. It tells me where to go"

"I think it would be under 'learn more about editing' because you're editing an article and you want to add a caption to an image."

Did try to click on the icon and the header at first, but then went to Getting started. Clicked Introduction to uploading images "Still not too clear on how to add a caption." Then went to the "Using an image" page. Missed it at first, but did then find the code to add a caption.

Found useful information for scenario: Yes

What frustrated you most about these pages? Why?: hard to find the info I was looking for on how to add caption to image, also had to be more proficient with HTML than I am
What did you like about the pages? Why?: I liked the second help page I saw with the larger print and the questions for categories better than the first help page I saw
Did you prefer the first or second page? Why?: as I said, I preferred the second page, it seemed easier to use and was more attractive to me
Do you have any other feedback?: I would refer people to basics about HTML or another website where they can get the basics of HTML

Image caption (2)

Scenario: "You want to figure out how to upload a picture to a Wikipedia article and you can’t figure out how to add a caption to the picture."

Test video

User info

Age: 51
Gender: Female
Experience: Average Web User
Daily Web Usage: 5-7 hours

Current page

"The first thing that I probably don't like, is I just feel overwhelmed with text." "I like that it gives me information underneath the links, so that it gives me a better clue what I'm going to find here." "I like the colours, I like the blues and the greys" "When I just focused in on the main area, I liked it."

For the scenario, the user thinks the info could be under Files, Technical information, or Frequently asked questions, but most likely Files "because it specifically talks about dealing with images". She clicked on Files.

She was slightly confused by the "Upload an image to Commons" link, since she was expecting help rather than a direct link to carry out an action. Links she thought might be helpful were Images and other uploaded files, How to insert and use pictures in Wikipedia articles and Guide to captions (although she noted that the latter was under the Policy and Guidelines header), and she found it difficult to decide between them.

On clicking How to insert and use pictures in Wikipedia articles her instant reaction was "ugh, this looks overwhelming". She found that captions were mentioned in the lede, but could not find them in the table of contents or within the page.

The user then tried the link Guide to captions instead. However this page is part of the Manual of Style, and describes what to write in a good caption, rather than how to add it. At this point she gave up and moved on to the next task.

Found useful information for scenario: No

New page

The user's instantaneous reaction to the new page was positive: "it just feels... nicer". She liked the icons and how "calming" the page was, but was less certain about which link to click. She eventually decided to click Getting started. On that page she quickly spotted and selected the Uploading images link (to one of the new tutorials).

She easily found the "Using an image" section, and was able to find out how to add a caption.

Found useful information for scenario: Yes

What frustrated you most about these pages? Why?: I was really frustrated by the lengthy content on the first set of pages. I never was able to find information that spoke directly about "adding a caption". I was overwhelmed by the amount of information to sift through. The second set of pages made me feel much calmer and not so overwhelmed!
What did you like about the pages? Why?: On the first set of examples I really liked the short info provided underneath the main category labels that gave me brief explanation on what I could expect to find if I clicked that link. Very helpful! The second set I just really liked how the content was greatly reduced! The first page was OK and provided a lot of separate links embedded within the paragraph describing the content. But my favorite part was when I clicked "Getting Started" and the content was very easy-to-scan, and there were clear labels on the left leading me to quickly find the area I needed to go to. It almost felt like a mini-web page.
Did you prefer the first or second page? Why?: Definitely the second! I felt less overwhelmed, I was able to find the information for which I was seeking (never did find it on the first!), there was less information to read through.
Do you have any other feedback?: When I finally did find the information on how to add a caption I was surprised that it could only be done via HTML (from what I could tell). I am not a programmer but did have one class in college so I know enough about HTML to spot what it is and could have figured out how to use it here. BUT, I'm not sure the average person could. I gave the example on video of a family blog I set up using "blogger" and, even though there are many family members who have very basic computer skills, a lot of my family members are able to post blogs, pictures, etc. So, I was surprised that, from what I saw, it didn't appear that a novice computer user could easily add to wikipedia. From what I've always heard about wikipedia this is just average people posting and adding to information. It seemed to me like it would require a bit of knowledge. But, maybe I just didn't see enough info