User:Zhanliusc/Results

Summary of Fall 2007 Study of Wikipedia and Citizendium

by Zhan Li, 1st year PhD student at the USC Annenberg School for Communication.

During the fall semester of 2007, I carried out a uses and gratifications-based study of the motivations of Wikipedia administrators and Citizendium editors (this is a class of Citizendium user which has administrative powers roughly comparable to Wikipedia administrators). The particular uses & gratifications framework used was based on:

Stafford, T., Stafford, M. and Schkade, L. (2004) Determining Uses and Gratifications for the Internet, Decision Sciences (journal), 35, 2, 259-285.

The framework defined three main kinds of motivation - social (gratifications from experiencing social relations), process (gratifications directly from the experience of doing things, similar to play, relaxation, leisure states), and content (gratifications from satisfying information/knowledge tasks such as finding out new facts, getting ideas etc.).

My study sought to find out if there were statistically significant differences between Wikipedia administrator and Citizendium editors along these three kinds of motivation in this framework. The statistical analysis employed was the independent samples t-test approach, with alpha set at .05 (any statistically significant differences are significant with 95% confidence that it is a true result).

The study was for an introductory statistical analysis class, and was my first ever experience with this kind of work. The resulting paper received an A minus (and I received an A minus for the course overall). Thanks again to all those who participated.

I am summarizing the results and and analysis of the study here.

Please note that all Wikipedia survey data that was initially obtained by an individual invitation recruitment method that was later objected to by several Wikipedia administrators was removed entirely from the study and was not subject to any analysis nor recording in results. The analysis and results relating to Wikipedia only use survey data collected through the alternative approach method approved by administrators.

Results

123 responses to the online survey were received in total. Out of these responses, 41 responses were found to be unusable due to response problems or invalid for group variable control reasons. There were 82 valid respondents in total, with 71 identifying themselves as Wikipedia administrators and 9 identifying themselves as Citizendium editors.

The mean age of the Wikipedia group was 31.15 (Standard Deviation (SD) = 11.86) with a range of 18 to 87. The gender breakdown was 87.7% male and 11.0% female with 1.4 % (1 respondent) declining to answer the gender question.

The mean age of the Citizendium group was 46.56 (SD = 13.24), with a range of 31 to 65. All members of this group were male.

Hypotheses

Two hypotheses (social and content) were found to be non-significant. One hypothesis (process) was found to be significant with a medium-to-large effect size.

Hypothesis 1 stated that the degree of social motivations will differ significantly between Wikipedia administrators and Citizendium editors. The difference in social motivations, t(78) = -.38, p = .71 (two-tailed) was found to be non-significant. The Wikipedia administrator participants had a mean score of 13.70 (SD = 4.67) and the Citizendium editors’ group had a mean score of 14.35(SD = 6.12). Hypothesis 1 was not supported.
Hypothesis 2 stated that the degree of process motivations will differ significantly between Wikipedia administrators and Citizendium editors. The difference in process motivations, t(76) = 2.14, p = .04 (two-tailed), was found to be significant. The Wikipedia administrators produced a higher mean for process motivations (M = 18.49, SD = 4.32) than the Citizendium editors (M = 14.88, SD = 6.10). As the finding was significant, this result supports Hypothesis 2. A medium to large effect size was found (d = .80, η2 = .06).
Hypothesis 3 stated that the degree of content motivations will differ significantly between Wikipedia administrators and Citizendium editors. The difference in content motivations, t(77) = -.60, p = .55 (two-tailed), was found to be non-significant. For content motivations, the Wikipedia group had a mean score of 24.94 (SD = 2.86) and the Citizendium group had a mean score of 25.56 (SD = 2.96). Hypothesis 3 was not supported.

Discussion

The purpose of this study has been to assess the hypotheses that there are significant differences in three categories of Internet U&G motivations (social, process and content) between Wikipedia administrators and Citizendium editors.

The analysis of the results showed that the data supported the hypothesis that there are significant differences in process motivations between Wikipedia administrators and Citizendium editors. On the other hand, there was no support for the hypotheses that there were significant differences in social and content motivations.

These conclusions would seem to correspond with philosophical similarities and differences claimed to exist between Wikipedia and Citizendium project cultures. Wikipedia administrators and Citizendium editors work in similar roles in open content projects that both aim to produce an authoritative encyclopedia and are based on the same kinds of social computing technologies. This suggests that their attitudes to the social and informational (content motivation related) context of their roles in their projects would be similar. However, Citizendium’s culture differs significantly in its striving for an expert-led, more formal atmosphere as opposed to the more informal and egalitarian Wikipedia culture. Citizendium’s signficantly lower score on the process motivations scale suggests that its editors are less tolerant of the idea that the participation in their project can (at least partly) be driven by leisure-based reasons of entertainment, relaxation and passing the time.

If these results were to be used to provide advice to those leading the Wikipedia and Citizendium projects, they may support an argument that those administrators/editors who individually have no preference over whether they work in a more formal or more informal online content environment are the ones who the projects are likely to compete the most over in hypothetical future recruitment efforts. Those who do have strong preferences are likely to self-select for one or the other project.

Scale Development

Item reliability analysis also suggested that specific U&G motivation scales (rather than general U&G Internet scales) should be developed for open content projects and for studies of Internet administrators.

Limitations

The main limitations were related to the unanticipated necessity of an alternative method (necessary to resolve the controversy over the original Wikipedia approach method) which reduced the reliability and validity of the study through a loss of control over randomization, sample size, equivalence of recruitment channels used in the two projects, and, in the Wikipedia case, the generation of some limited local publicity. Given these problems, conclusions about the Internet U&G motivations approach and the differences between the Wikipedia and Citizendium groups are of limited reliability and power. Improved future versions of this study would have a better planned recruitment approach where the researcher would be more well-informed about likely community preferences. A better balance of samples between the two projects would also be a welcome enhancement. Furthermore, as this study focussed on two specific online open content encyclopedia projects, the conclusion may have limited generalizability to other encyclopedia and open content projects.

Any further questions can be addressed to zhanli at usc dot edu

Thanks, and Happy Holidays! Zhanliusc (talk) 02:53, 24 December 2007 (UTC)