User:Kokiri/WQA2
Two years since my last attempt (WQA 1) to assess the quality of the Pedia, here comes an update. On the one hand, I have included the same measures as last time, allowing for a comparison where possible. On the other hand, I have included a few extra measures. Please feel free to edit this page if you feel the presentation could be improved.
The other quality assessment I am aware of was by Adam Carr, carried out in October 2003: English Wikipedia Quality Survey.
I have sampled 275 articles on 18 December 2005, using the random article function. This means, in contrast to WQA 1, WQA 2 uses a representative random sample with a confidence level of 90% and a margin of error of 5%. I have not assessed the quality of the articles themselves, something Nature have recently done with their non-representative sample. I have also not assessed the level of plagiarism that I expect the Pedia to be guilty of.
Definitions
[edit]entry: what is commonly referred to a an article on the Pedia, not regarding its length or quality
article: a proper entry in the Pedia, not a stub or any other category listed here; articles can be quite short according to the criteria I applied
bot: a bot generated article; in practice this refers to Rambot articles
fragment: an entry with the structure of an article, but mostly consisting of titles and other non-content (called gaps in WQA 1)
list: an entry that is just a list links
disambiguation: disambiguation page
stub: a stub
spam: I came across one entry that was spam (which I flagged for speedy deletion)
Overview: Crude statistics
[edit]I guess that most people will be interested in the following table. Bear in mind the margin of error (5%) and the confidence level (90%) throughout the analysis.
Quality (ordered) | % | Plot |
Spam | .4 | |
Stub | 40.0 | ******** |
List | 12.0 | ** |
Fragment | 9.5 | ** |
Bot | 4.0 | * |
Articles | 34.2 | ******* |
This means that only 34.2% of all Wikipedia entries are articles of some sort.
Compared to WQA 1 two years ago
[edit]Two years ago, in my first assessment I came up with the following numbers:
Category | # | % | Plot |
Article | 19 | 38 | ******** |
Bot | 12 | 24 | ***** |
Fragment | 4 | 8 | ** |
List | 3 | 6 | * |
Stub | 12 | 24 | ***** |
Bear in mind the smaller sample (N=50) back then.
Frequencies of article features
[edit]What follows is a bunch of frequency tables on some of the variables in WQA 2. These are features that any article and stub can have. A low percentage of entries with a certain feature does not necessarily indicate poor quality: not all features are equally desirable in all articles.
Categories
[edit]Two years ago, we did not have categories. Now almost every article and stub is categorized in some way.
Categories | Count | % | Plot |
0 | 18 | 6.5 | * |
1 | 73 | 26.5 | ***** |
2 | 94 | 34.2 | ******* |
3 | 39 | 14.2 | *** |
4 | 26 | 9.5 | ** |
5 | 11 | 4 | * |
6 | 2 | .7 | |
7 | 3 | 1.1 | |
8 | 4 | 1.5 | |
9 | 1 | .4 | |
10 | 3 | 1.1 | |
14 | 1 | .4 | |
Total | 275 |
Formulae
[edit]A rare sighting.
0 Formulae | 273 | 99.3% | ******************** |
1 Formula | 1 | .4% | |
3 Formulae | 1 | .4% |
Maps
[edit]Not very common, but then again, not every article should have one. Still, many place articles do not have a map. I counted 29 entries on places, and 13 maps...
0 Maps | 262 |
1 Map | 13 |
Pictures and Illustrations
[edit]Most articles still come without illustration. I have not come across any animations or videos, something other encyclopaedias brag with... I know they exist, but the fact that they do not show in this assessment suggests that there are not many of them.
Frequency | # | % | Plot |
0 | 216 | 78.5 | **************** |
1 | 39 | 14.2 | ** |
2 | 12 | 4.4 | * |
3 | 4 | 1.5 | |
4 | 1 | .4 | |
6 | 1 | .4 | |
9 | 1 | .4 | |
20 | 1 | .4 | |
Total | 275 | 100.0 |
References
[edit]At least we have now started with references, but they are largely absent. This not only compromises the verifiability of entries, but is probably a sign of rampant plagiarism (rephrasing something does not mean the source should be attributed).
Frequency | # | % | Plot |
0 | 254 | 92.4 | ****************** |
1 | 10 | 3.6 | * |
2 | 5 | 1.8 | |
3 | 1 | .4 | |
4 | 4 | 1.5 | |
5 | 1 | .4 | |
Total | 275 | 100.0 |
Tables
[edit]As with illustrations, not very common. There is of course the question whether tables should be used in some entries at all, because they are not always a useful way to summarize information.
0 Tables | 227 | 82.5% | **************** |
1 Table | 44 | 16% | *** |
2 Tables | 4 | 1.5% |
Length
[edit]Again, like two years ago, I have used a rather convenient way to measure the length of entries.
Frequency | Count | % | Plot |
3 lines or less | 69 | 25.1 | ***** |
less than 1 screen | 77 | 28.0 | ****** |
1 Screen | 29 | 10.5 | ** |
2 Screens | 51 | 18.5 | **** |
3 Screens | 26 | 9.5 | ** |
Longer | 23 | 8.4 | ** |
Britannica comparison
[edit]As last time, I have checked whether the entry was also in Britannica. Last time I used the 2002 DVD version of Britannica; this time I used search.eb.com, so in Britannica also means in the 'Britannica Student Encyclopaedia. Not in Britannica means that there were no matches, In Britannica means that there is an article with the same or an equivalent title, Within Wider Britannica Article means that the topic is treated within a Britannica article that covers a larger topic. This is possibly my pet peeve: many stubs have little potential to grow because they would better be dealt with in a more general article (see WQA 1).
Not in Britannica | 166 | 60.4% | ************ |
In Britannica | 32 | 11.6% | ** |
Within Wider Britannica Article | 57 | 20.7% | **** |
Areas covered
[edit]Here are two tables on the areas the entries cover. First I included a category for persons, then I split this category to the others. So, in the first table, a politician would be counted as person, in the second she or he would be found under politics. The areas are to a large extent influenced by the articles, and do not follow an existing categorization. This must be borne in mind when considering the systematic bias of the Pedia.
Including persons
[edit]Area | Count | % |
Animal | 6 | 2.2 |
Architecture | 4 | 1.5 |
Company | 3 | 1.1 |
Culture | 4 | 1.5 |
Disambiguation | 20 | 7.3 |
Geography | 10 | 3.6 |
History | 6 | 2.2 |
Language | 2 | .7 |
Law | 5 | 1.8 |
Leisure | 12 | 4.4 |
Literature | 14 | 5.1 |
Media | 12 | 4.4 |
Music | 14 | 5.1 |
Nobility | 6 | 2.2 |
Organization | 9 | 3.3 |
Person | 55 | 20.0 |
Place | 30 | 10.9 |
Plant | 2 | .7 |
Internet | 4 | 1.5 |
Religion | 4 | 1.5 |
Science | 13 | 4.7 |
Sports | 10 | 3.6 |
Standard | 2 | .7 |
Technology | 4 | 1.5 |
Misc. | 5 | 1.8 |
Transport | 7 | 2.5 |
Other | 6 | 2.2 |
Warfare | 6 | 2.2 |
Total | 275 | 100.0 |
Persons assigned to other areas
[edit]Area | Count | % |
Animal | 5 | 1.8 |
Art | 15 | 5.5 |
Company | 3 | 1.1 |
Culture | 6 | 2.2 |
Disambiguation | 20 | 7.3 |
Fiction | 6 | 2.2 |
Geography | 10 | 3.6 |
History | 6 | 2.2 |
Language | 2 | .7 |
Law | 5 | 1.8 |
Leisure | 12 | 4.4 |
Literature | 12 | 4.4 |
Media | 15 | 5.5 |
Music | 22 | 8.0 |
Nobility | 6 | 2.2 |
Organization | 9 | 3.3 |
Place | 29 | 10.5 |
Plant | 2 | .7 |
Internet | 4 | 1.5 |
Politics | 10 | 3.6 |
Religion | 9 | 3.3 |
Science | 17 | 6.2 |
Sports | 18 | 6.5 |
Standard | 2 | .7 |
Technology | 4 | 1.5 |
Misc. | 5 | 1.8 |
Transport | 7 | 2.5 |
Other | 6 | 2.2 |
Warfare | 8 | 2.9 |
Total | 275 | 100.0 |
Predicting what is an article
[edit]The following table is the result of a regression analysis, trying to predict what makes an article (as opposed to a stub, fragment, or the like). Hits in Google and Scirus are insignificant predictors, meaning that some articles have many Google hits, others just a few. If the topic occurs in Britannica, it is 1.5 times as likely to be an article as a topic that does not. This, however, is not statistically significant once the number of incoming links is considered. The number of links to an entry is the single most powerful predictor whether an entry will be an article or anything less: every incoming link increases the chance of being an article by 8%.
Sig. | Exp(B) | |
Is in EB | .298 | 1.418 |
Google Hits | .936 | 1.000 |
Scirus Hits | .768 | 1.000 |
Scirus Journal Hits | .446 | 1.000 |
Links In | .000 | 1.078 |
Constant | .000 | .171 |
I have also run the regression with splitting the different kinds of Britannica entries (equivalent or within other article). Again, the number of Google hits is irrelevant. The number of incoming links is a very good predictor. Moreover, we can see why the distinction between the kinds of Britannica entry is important: a Wikipedia entry with an equivalent article in Britannica is 2.3 times as likely to be an article than an entry without. Wikipedia entries with an article in Britannica that covers a wider topic do not fare significantly better than entries that have no entry in Britannica at all. I take this as a sign that we have too many entries that have no potential to grow...
Sig. | Exp(B) | |
1000 Google Hits | .734 | 1.000 |
EB Equivalent | .102 | 2.305 |
EB within Larger | .605 | 1.216 |
Links In | .000 | 1.076 |
Constant | .000 | .175 |
Predicting what is a stub
[edit]Here are the results of a regression analysis that predicts whether an entry is a stub as opposed to anything else. We find the opposite here. Again, the number of hits in Google and Scirus are insignificant. The number of incoming links is significant: For every extra incoming link, the entry is 8% less likely to be a stub. If there is an entry in Britannica, the chances of being a stub drop by 34.7%.
Sig. | Exp(B) | |
Is in EB | .189 | .653 |
Google Hits | .792 | 1.000 |
Scirus Hits | .774 | 1.000 |
Scirus Journal Hits | .979 | 1.000 |
Links In | .000 | .912 |
Constant | .000 | 2.253 |
Predicting article length
[edit]This table summarizes the prediction of article length. All these predictors are statistically significant (.1 level), with the exception of the number of hits in Scirus. An entry which can also be found in Britannica is expected to be about half a screen larger than one that cannot. The effects of hits in the search engines are significant but very small. It takes 1 million Google hits to increase the article length by half a screen (0.000000492 screens for every Google hit); or about 5000 Scirus Journal hits for the same effect (0.0000935 screens for every Scirus Journal hit).
B | Sig. | |
(Constant) | 1.062 | .000 |
Is in EB | .582 | .002 |
Hits in Google | 4.92E-007 | .004 |
Hits in Scirus | -2.02E-005 | .117 |
Journal Hits in Scirus | 9.35E-005 | .092 |
Features in longer articles
[edit]Longer articles tend to have more of the features measured. All correlations are positive and significant at the .01 level. The number of incoming links once again shows up as the strongest effect.
Number of Maps | .172 ** |
Number of Pictures | .172 ** |
Number of References | .179 ** |
Number of Tables | .236 ** |
Links to Entry | .473 ** |
The data
[edit]Feel free to make use of the data for your own analyses. Bear in mind the predictive limits outlined in the introduction: Data.