Jump to content

Wikipedia:Wikipedia Signpost/2016-11-04/Recent research

Add links
From Wikipedia, the free encyclopedia
The Signpost
← Back to Contents
View Latest Issue


Recent research

Why women edit less, and where they are overrepresented; article importance and quality; predicting elections from Wikipedia

Contribute  —  
Share this
By Giuseppe Profiti, Tilman Bayer, Morten Warncke-Wang, Jonathan Morgan, and Zareen Farooqui

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

"Gender gap on Wikipedia: visible in all categories?"

Reviewed by Giuseppe Profiti
Asteroids are among the categories with the most overrepresentation of male editors, and figure skating among those with most female overrepresentation

This bachelor thesis[1] looks for gender imbalance among editors for specific categories in the English Wikipedia. The analysis is based on the edits of users who publicly disclosed their gender (about 176 thousand) to more than 3.7 million articles in 470 categories (derived from DBpedia's ontology, rather than Wikipedia's inbuilt category system). The thesis first establishes the distribution of editors by gender (roughly 85% males and 15% females). The number of edits by each group is statistically compared to that baseline distribution. For each category, if it varies from the baseline, it is considered to represent a gender gap, i.e. that editors from that gender are overrepresented in that category.

The results show that despite the huge imbalance in the two groups, pages in some categories receive more edits from users belonging to one gender, while other categories are dominated by the other one. As the "Top five categories where male editors are most overrepresented", the author lists "YearInSpaceflight", "Asteroid", "BaseballSeason", "MotorsportSeason", and "FormulaOneTeam". He observes sports as recurring theme "throughout all significant 'male categories'. Besides sports other recurring subjects are transport and politics." On the other hand, "the categories with a female overrepresentation show somewhat less obvious recurring themes. Many of these categories are more or less culture related however." The five categories with the most female overrepresentation are "FigureSkater", "Skater", "Garden", "GaelicGamesPlayer", and "Mollusca".

While highlighting some information on such unbalanced distribution, the underlying hypothesis could be further explored by using the quantity of text changed in each edit and other patterns mentioned by the author.

(See related Signpost coverage from 2011: "New tool analyzes article contributors' gender and location")

Quality and importance in different language editions

Reviewed by Morten Warncke-Wang

While much is known about the quality of Wikipedia articles, less is known about how the different language editions assess article importance. The English Wikipedia's article about waffles is for instance labelled "top-importance" by WikiProject Breakfast, the highest category possible, but at the same time labelled "high importance" by WikiProject Food and Drink (you can find both of these labels on waffle’s talk page). A paper at the International Conference on Information and Software Technologies studies titled "Quality and Importance of Wikipedia Articles in Different Languages"[2] studies the connection between importance and quality. The paper's three research questions look at whether importance affects quality, what parameters are useful for applying machine learning to automatically assess importance, and if there are differences between how language editions model importance.

The English edition offers the most data on article importance, and the paper therefore uses a dataset of English articles to test if importance affects quality. Using a random forest classifier and a model with 85 parameters, a modest increase in classifier performance is found when importance is added as a parameter, indicating that importance affects quality. The same dataset and model is then trained to predict article importance, finding that about two-thirds of top- and low-importance articles can be correctly identified. Lastly the paper compares the importance of model features between different language editions, finding many differences, although these are not described in more detail.

Research on aspects of article quality across different language editions is an area that has not received a lot of attention, making this paper a welcome addition to the literature. It is also great to see article importance being studied. At the same time, this paper could have made a much stronger contribution through comparisons against a sensible baseline (this reviewer notes that the paper cites an in-press paper by the same authors[supp 1], although that paper's results do not appear to be available in English) because the classifier performance appears to be similar to for instance ORES although ORES uses a model with a lot fewer parameters. A deeper investigation into article importance would also be worthwhile, for example because importance differs between topic areas, as exemplified by the article on waffle described earlier.

Why women edit less: a controlled experiment

Reviewed by Jonathan Morgan

Researchers have attempted to quantify Wikipedia's gender gap and its impact on content type and quality, and to understand the reasons for the gender gap. A new journal article[3] attempts to experimentally evaluate several hypotheses for why women tend to edit Wikipedia less than men do.

The researchers asked 192 male and female college students to contribute a draft essay about school bullying. The version of the draft that participants were asked to work on had already been edited by four other users (secretly, the researchers themselves), identified by pseudonyms. Two of the pseudonyms were obviously gendered ("Ms Trouble", "Mr Football"), and two were gender neutral ("Cheerios4Life", "AnonymousOne"). Since most people are not familiar with the mechanics of wiki editing, the researchers used a Microsoft Word document with "track changes" enabled as a platform for the editing task, to simulate the versioning and commenting capabilities of MediaWiki pages. The researchers also surveyed the students to gather relevant demographic and psychometric data, and compared their survey responses with their editing behaviors.

Findings from this study include that while women edited more than men overall (contributed more words to the draft), they were less likely to edit under the conditions designed to approximate the social environment of Wikipedia. Specifically, women edited less where there were few or no female-identified collaborators present, and where feedback from the pseudonymous collaborators was neutral (vs. constructive). Interestingly, female participants also tended to assume that one of the non-gendered pseudonyms ("AnonymousOne") was male, and also evaluate feedback from that editor as more critical than male participants who received the same feedback. Based on these findings, the researchers suggest that increasing the visibility of female editors and encouraging constructive feedback may encourage more women to edit Wikipedia.

"Wikipedia traffic data and electoral prediction: towards theoretically informed models"

Reviewed by Zareen Farooqui

This research[4] aims to explore the relationship between Wikipedia page view statistics and electoral results during the 2009 and 2014 European Parliament elections in regards to overall voter turnout and individual party results. The article suggests two reasons why voters might seek information: to research new parties which are beyond the voter's familiarity, and to research alternative party options if a voter is unhappy with the party they previously supported (thus becoming swing voters).

The first dataset used in this research is Wikipedia page views data on the general election page in 14 different languages (those which are the primary languages of the voting countries). The second dataset includes political parties which had at least 5% vote share in the 2009 and/or 2014 elections in the UK, France, Germany, Spain and Italy. The researchers gathered additional data points such as number of views to the political party's Wikipedia page the week before the election, the final percentage of vote share each party received, whether a party was new, whether a party was incumbent, and the number of times each party was mentioned in print media during the week before the election.

Comparing the relative change in page views to the EU Parliament elections article and total voter turnout in the 2009 and 2014 elections indicates that interest in election events is proportional to levels of readership on Wikipedia. This research suggests that often the party garnering the most page views does not win the election, rather, it may be a smaller party which interested swing voters. Figure 1(a) shows a high correlation between print media mentions and overall voter share for parties. Figure 1(b) shows Wikipedia page views may predict a new party's success, while news outlet mentions are better at predicting an established party's success.

News media mentions compared with (a) Wikipedia page views and (b) absolute level of vote share

The research tests the theory that an increase in Wikipedia page views may suggest an increase to votes for a party using three linear ordinary least squares regression models. The first model is a baseline of past voting results. The second model is also a baseline model which includes past voting results, along with all other non-Wikipedia related data collected. These baseline models serve as a comparison to the third model, which includes all the previously modeled data, along with two Wikipedia-related parameters. The models show that Wikipedia can be considered a predictor of voter outcome, but it only marginally improves upon the baseline models. Wikipedia's predictive power lies in predicting the amount a party's vote share may increase or decrease from the previous election cycle.

As noted by the researchers, one limitation of this article is that the data is at an aggregated level, while all theories are at the micro level. Also, it is unclear what number of Wikipedia page views reflect voters versus other groups, such as journalists or those those affiliated with the parties.

(See also our 2014 coverage of some related blog posts by the same authors: "Wikipedia use driven by news media or replacing news media?")

Briefly

Conferences and events

See the research events page on Meta-wiki for upcoming conferences and events, including submission deadlines.

Other recent publications

Other recent publications that could not be covered in time for this issue include the items listed below. contributions are always welcome for reviewing or summarizing newly published research.

  • "Disinformation on the Web: impact, characteristics, and detection of Wikipedia hoaxes"[5] From the abstract: "We find that, while most hoaxes are detected quickly and have little impact on Wikipedia, a small number of hoaxes survive long and are well cited across the Web. Second, we characterize the nature of successful hoaxes by comparing them to legitimate articles and to failed hoaxes that were discovered shortly after being created. We find characteristic differences in terms of article structure and content, embeddedness into the rest of Wikipedia, and features of the editor who created the hoax. Third, we successfully apply our findings to address a series of classification tasks, most notably to determine whether a given article is a hoax. And finally, we describe and evaluate a task involving humans distinguishing hoaxes from non-hoaxes. We find that humans are not good at solving this task and that our automated classifier outperforms them by a big margin."
  • "Where are the women in Wikipedia? Understanding the different psychological experiences of men and women in Wikipedia"[6] From the abstract: "We analyzed data from a sample of 1,598 individuals in the United States who completed the English version of an international survey of Wikipedia users and readers conducted in 2008 and who reported being occasional contributors. ... Women reported less confidence in their expertise, expressed greater discomfort with editing (which typically involves conflict) and reported more negative responses to critical feedback compared to men. Mediation analyses revealed that confidence in expertise and discomfort with editing partially mediated the gender difference in number of articles edited, the standard measure for contribution to Wikipedia." (See also our 2012 coverage of a related paper by the same authors: "Gender gap connected to conflict aversion and lower confidence among women")
  • "Wikipedia and stock return: Wikipedia usage pattern helps to predict the individual stock movement"[7] From the abstract: "We provide evidence that data on how often a company’s Wikipedia page is being viewed is linked to its subsequent performance in the stock market. We then develop a portfolio in line with the Wikipedia usages and demonstrate that our investment strategy based on Wikipedia views is profitable both financially and statistically."
  • "Editing diversity in: reading diversity discourses on Wikipedia"[8] From the abstract: "... the Wikimedia Foundation (WMF) has devoted a fair amount of time and resources to tackling [Wikipedia's] ‘gender gap.’ While we acknowledge the good intentions of the WMF and volunteer efforts to improve conditions for women editors on Wikipedia, we argue that borrowing from corporatized diversity initiatives more effectively supports organizational growth rather than addresses the underlying reasons behind women’s low representation and participation."
  • "Circadian patterns on Wikipedia edits"[9] From the abstract: "We ... show in this work that Wikipedia editing presents well defined periodic patterns with respect to daily, weekly and monthly activity. In addition, we also show the periodic nature of the number of inter-event in time."
    From the rest of the paper: "Our data sample is a database of WP edits, of pages written in English in the period of about 10 years ending in January 2010 ... In general, [the 100 most active] editors have the main power peak at ∼ 1.157 × 10−5 Hz corresponding to a period of 24 h and a second peak at ∼ 2.315 × 10−5 Hz, matching a 12 h period, a harmonic from the main frequency. ... The highest activity peak can switch between mornings and evenings, depending on the day. In the process of WP editing, the change of activity patterns on week-ends is clear. ... Along the year the intensity of activity seems conditioned by holidays."
    See also our 2011 coverage of a related paper: "Wikipedians' weekends in international comparison"
  • "Computational fact checking from knowledge networks"[10] From the abstract: "we show that the complexities of human fact checking can be approximated quite well by finding the shortest path between concept nodes under properly defined semantic proximity metrics on knowledge graphs. Framed as a network problem this approach is feasible with efficient computational techniques. We evaluate this approach by examining tens of thousands of claims related to history, entertainment, geography, and biographical information using a public knowledge graph extracted from Wikipedia. Statements independently known to be true consistently receive higher support via our method than do false ones."
  • "Challenges of mathematical information retrieval in the NTCIR-11 Math Wikipedia Task"[11] From the abstract: "... the optional Wikipedia Task provides a test collection for retrieval of individual mathematical formula from Wikipedia based on search topics that contain exactly one formula pattern. We developed a framework for automatic query generation and immediate evaluation."
  • "Quantifying the relationship between hit count estimates and Wikipedia article traffic"[12][predatory publisher] From the abstract: "This paper analyzes the relationship between search engine hit counts and Wikipedia article views by evaluating the cross correlation between them. We observe the hit count estimates of three popular search engines over a month and compare them with the Wikipedia page views. The strongest cross correlations are recorded with their delays in days."
  • "LeadWise: using online bots to recruite and guide expert volunteers"[13] From the abstract: "we propose LeadWise, a system that uses social media bots to recruit and guide contributions from experts to assist non-profits in reaching their goals. ... We focus in particular on experts who can help Wikipedia in its objective of reducing the gender gap by covering more women in its articles. Results from our first pilot show that LeadWise was able to obtain a noteworthy number of expert participants in a two week period with limited requests to targeted specialists."
    From the rest of the article: "We created 'CauseBots' [on Twitter] which are bots that present themselves as a social cause (hiding that the accounts are an automated agent). We also created 'AgentBots' which are bots that present themselves as bots supporting a social cause. ... the first thing all of LeadWise’s bots do is build a 'supportive audience' with experts. ... Once [they have] a supportive audience with over fifteen members, the bots follow the same behavioural rules to request and guide participation: They publicly ask for the names of women who should be added to Wikipedia. ... We primarily focused on Spanish speaking experts in gender equality. ... We considered that experts were individuals who tweeted heavily about gender equality. Both bots looked for users mentioning related Spanish keywords, such as 'equidad de genero,' and who had already published a large number of related tweets (over 50). ... In total, 22 new women were added [by these experts recruited on Twitter] to the list of Wikipedia articles to cover." TB

References

  1. ^ Schrijver, Paul (2016-05-25). Gender gap on Wikipedia: visible in all categories? (Thesis). University of Amsterdam. (bachelor thesis)
  2. ^ Lewoniewski, Włodzimierz; Węcel, Krzysztof; Abramowicz, Witold (2016-10-13). "Quality and Importance of Wikipedia Articles in Different Languages". In Giedre Dregvaite; Robertas Damasevicius (eds.). Information and Software Technologies. Communications in Computer and Information Science. Vol. 639. Springer International Publishing. pp. 613–624. doi:10.1007/978-3-319-46254-7_50. ISBN 9783319462547.
  3. ^ Shane-Simpson, Christina; Gillespie-Lynch, Kristen (2016-10-06). "Examining potential mechanisms underlying the Wikipedia gender gap through a collaborative editing task". Computers in Human Behavior. 66 (January 2017): 312–328. doi:10.1016/j.chb.2016.09.043. Closed access icon
  4. ^ Yasseri, Taha; Bright, Jonathan (2016-06-18). "Wikipedia traffic data and electoral prediction: towards theoretically informed models". EPJ Data Science. 5 (1). doi:10.1140/epjds/s13688-016-0083-3. S2CID 256241960.
  5. ^ Kumar, Srijan; West, Robert; Leskovec, Jure (2016). "Disinformation on the Web: impact, characteristics, and detection of Wikipedia hoaxes" (PDF). Proceedings of the 25th International World Wide Web Conference. WWW 2016. doi:10.1145/2872427.2883085. ISBN 978-1-4503-4143-1.
  6. ^ Bear, Julia B.; Collier, Benjamin (2016-01-04). "Where are the women in Wikipedia? Understanding the different psychological experiences of men and women in Wikipedia". Sex Roles. 74 (5–6): 254–265. doi:10.1007/s11199-015-0573-y. S2CID 255003641. Closed access icon Author's copy (free account required)
  7. ^ Wei, Pengyu; Wang, Ning (2016). "Wikipedia and stock return: Wikipedia usage pattern helps to predict the individual stock movement" (PDF). Proceedings of the 25th International Conference Companion on World Wide Web. WWW '16 Companion. Republic and Canton of Geneva, Switzerland: International World Wide Web Conferences Steering Committee. pp. 591–594. ISBN 9781450341448.
  8. ^ MacAulay, Maggie; Visser, Rebecca (2016-05-01). "Editing diversity in: reading diversity discourses on Wikipedia". Ada: A Journal of Gender, New Media, and Technology.
  9. ^ Gandica, Y.; Lambiotte, R.; Carletti, T.; Aidos, F. Sampaio dos; Carvalho, J. (2016-03-06). "Circadian Patterns on Wikipedia Edits". In Hocine Cherifi; Bruno Gonçalves; Ronaldo Menezes; Roberta Sinatra (eds.). Complex Networks VII. Studies in Computational Intelligence. Vol. 644. Springer International Publishing. pp. 293–300. doi:10.1007/978-3-319-30569-1_22. ISBN 9783319305684. Closed access icon
  10. ^ Ciampaglia, Giovanni Luca; Shiralkar, Prashant; Rocha, Luis M.; Bollen, Johan; Menczer, Filippo; Flammini, Alessandro (2015-06-17). "Computational fact checking from knowledge networks". PLOS ONE. 10 (6): e0128193. arXiv:1501.03471. Bibcode:2015PLoSO..1028193C. doi:10.1371/journal.pone.0128193. PMC 4471100. PMID 26083336.
  11. ^ Schubotz, Moritz; Youssef, Abdou; Markl, Volker; Cohl, Howard S. (2015). "Challenges of mathematical information retrieval in the NTCIR-11 Math Wikipedia Task". Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR '15. New York, NY, USA: ACM. pp. 951–954. doi:10.1145/2766462.2767787. ISBN 978-1-4503-3621-5. Closed access icon
  12. ^ Tian, Tina; Agrawal, Ankur (2015). "Quantifying the relationship between hit count estimates and Wikipedia article traffic". International Journal of Advanced Computer Science and Applications. 6 (5). doi:10.14569/IJACSA.2015.060504.
  13. ^ Flores-Saviaga, Claudia; Savage, Saiph; Taraborelli, Dario (2016). "LeadWise: using online bots to recruite and guide expert volunteers". Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion. CSCW '16 Companion. New York, NY, USA: ACM. pp. 257–260. doi:10.1145/2818052.2869106. ISBN 978-1-4503-3950-6. Closed access icon Author's copy
Supplementary references:
  1. ^ Lewoniewski, Włodzimierz; Węcel, Krzysztof; Abramowicz, Witold (2015). Analiza porównawcza modeli jakości informacji w narodowych wersjach Wikipedii.
← Previous "Recent research"
Next "Recent research" →
+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Why women edit less

Study after study by researchers, sitting in ivory towers, about the causes of the gender gap on Wikipedia, yet none have studied what is obvious to many of the foot-soldiers edting here.

For example: How about the use of derogatory language when it comes to depicting women on Wikipedia referring to them females. This has been brought up time and again, but is still dismissed as silliness by many, including editors who themselves belong to a marginalized real-life group such as Gay (for example).

And how about marginalizing those who support more inclusiveness of women. It is not unheard of to characterize such editors who are perceived to be men as ‘’creepy’’. It creates an environment where any editor who is the target of attacks (many/most editors here) must think twice before showing support for these issues. Ottawahitech (talk) 18:43, 5 November 2016 (UTC)please ping me[reply]

Sure thing, colleague. And other male pigs derogate innocent maidens referring to them women. Staszek Lem (talk) 00:45, 8 November 2016 (UTC)[reply]
@Staszek Lem: Could you please elaborate: I don't understand the point you are trying to make. Thanks in advance, Ottawahitech (talk) 01:12, 9 November 2016 (UTC)please ping me[reply]
@Ottawahitech: I find it weird that some find the word "wikt:female" derogatory, which is no less weird that the same some find the word "woman" derogatory as well, since it is etymologically inferior to "man". Staszek Lem (talk) 01:51, 9 November 2016 (UTC)[reply]
@Staszek Lem: hmmmm…As far as using the word female to describe female humans there are plenty of RSes (just google it) that find it to be derogatory. This is what I found on the top of the google search for "woman vs female" today. It is news to me that some people find the word "woman" derogaroy, but I am learning new things every day on Wikipedia. Ottawahitech (talk) 16:08, 9 November 2016 (UTC)please ping me[reply]
@Ottawahitech: Yep, it is amazing to what height of idiotism political correctness may be elevated. Last thing I learned some Chinese people find it insulting when being referred to as "that Chinese guy" At first "Oriental" became derogatory, next "Asian" became an insult. And now "Chinese". It comes both from inferiority complex and social stigma/stereotyping. I know it firsthand: when an American calls me "a Pole" I bet my ancestor's szabla he is thinking "Polack joke". Do you remember a scandal in the media about wikipedia when some militant feminist noticed the category:Women writers? To label someone a "woman writer" was fussed as an extremely gross insult. Staszek Lem (talk) 03:25, 10 November 2016 (UTC)[reply]
@Staszek Lem: Yes I remember more than one scandal in the media that involved the depiction of women on Wikipedia. I believe the one you are referring to is the Amanda Filipacchi controversy, which was not about labeling "someone a woman writer”" but about the fact that articles about women were being deliberately "orphaned" by placing them in categories that contained only women and removing articles about them from mainstream categories. It became a Wikipedia scandal when it was revealed by Filipacchi that experienced editors were behind this massive dislocation of articles about women. It was a scandal because the editors responsible were knowingly doing this in contravention of Wikipedia guidelines. Ottawahitech (talk) 17:17, 11 November 2016 (UTC)please ping me[reply]
@Ottawahitech:the text says "perceived minimization of female novelists", i.e., labeling women as women is "minimalization", right? Pray tell me which guideline was violated. We routinely create subcategory by defining characteristics. Now, what shall you do with the ""racist"" Category:American women novelists of Indian descent? Some curry-muncher hater barred Indu Sundaresan from Category:American women novelists. Must be Donald Trump's hand. Staszek Lem (talk) 18:18, 11 November 2016 (UTC)[reply]
@Staszek Lem: the guideline which was deliberately violated by experienced editor(s) who removed hundreds of articles about women writers from sub/categories of Category: Writers is Wikipedia:GHETTO. This all happened around April 1, 2013. Ottawahitech (talk) 16:19, 14 November 2016 (UTC)please ping me[reply]
@Ottawahitech:I admit I have never seen this guideline. Unfortunately our rulebook grew to become extremely unwieldy, so I am pretty much sure that it was not violated "deliberately". After seeing the guideline, I disagree with many of its guidance. In particular, the advice "do not create a subcategory for "African-American poets" is highly dubious, because there are tons of books of literary criticism specifically about AA poets. It is a well-defined and extremely useful category. Of course, some may argue that its function may be served by List of African-American poets, but it is long established that lists and categories are different, complementary search tools. Still, I agree with its core, but for a different reason: "ghettoisation" into minuscule subcategories makes search in categories extremely inconvenient. But this is not restricted to race and gender. For example, try to find an American city I forgot its name, ony remember it starts with "Ap". But this is the problem of WMF, which wastes its money on various weird projects instead of improving the wikimedia engine. Staszek Lem (talk) 18:00, 14 November 2016 (UTC)[reply]

I never thought of molluscs as having some kind of female connotations. Or asteroids as male - they're just big rocks in space, what could be less gendered? Maybe there is something I am missing.--Bellerophon5685 (talk) 06:40, 11 November 2016 (UTC)[reply]

@Bellerophon5685: are you saying that the gender of wikipedia editors is not important?Ottawahitech (talk) 19:50, 24 November 2016 (UTC)please ping me[reply]
Explore Wikipedia history by browsing The Signpost archives.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia:Wikipedia_Signpost/2016-11-04/Recent_research&oldid=1193876553"