Jump to content

User:Kokiri/WQA2

From Wikipedia, the free encyclopedia

Two years since my last attempt (WQA 1) to assess the quality of the Pedia, here comes an update. On the one hand, I have included the same measures as last time, allowing for a comparison where possible. On the other hand, I have included a few extra measures. Please feel free to edit this page if you feel the presentation could be improved.

The other quality assessment I am aware of was by Adam Carr, carried out in October 2003: English Wikipedia Quality Survey.

I have sampled 275 articles on 18 December 2005, using the random article function. This means, in contrast to WQA 1, WQA 2 uses a representative random sample with a confidence level of 90% and a margin of error of 5%. I have not assessed the quality of the articles themselves, something Nature have recently done with their non-representative sample. I have also not assessed the level of plagiarism that I expect the Pedia to be guilty of.

Definitions

[edit]

entry: what is commonly referred to a an article on the Pedia, not regarding its length or quality

article: a proper entry in the Pedia, not a stub or any other category listed here; articles can be quite short according to the criteria I applied

bot: a bot generated article; in practice this refers to Rambot articles

fragment: an entry with the structure of an article, but mostly consisting of titles and other non-content (called gaps in WQA 1)

list: an entry that is just a list links

disambiguation: disambiguation page

stub: a stub

spam: I came across one entry that was spam (which I flagged for speedy deletion)

Overview: Crude statistics

[edit]

I guess that most people will be interested in the following table. Bear in mind the margin of error (5%) and the confidence level (90%) throughout the analysis.

Quality (ordered) % Plot
Spam .4
Stub 40.0 ********
List 12.0 **
Fragment 9.5 **
Bot 4.0 *
Articles 34.2 *******

This means that only 34.2% of all Wikipedia entries are articles of some sort.

Compared to WQA 1 two years ago

[edit]

Two years ago, in my first assessment I came up with the following numbers:

Category # % Plot
Article 19 38 ********
Bot 12 24 *****
Fragment 4 8 **
List 3 6 *
Stub 12 24 *****

Bear in mind the smaller sample (N=50) back then.

Frequencies of article features

[edit]

What follows is a bunch of frequency tables on some of the variables in WQA 2. These are features that any article and stub can have. A low percentage of entries with a certain feature does not necessarily indicate poor quality: not all features are equally desirable in all articles.

Categories

[edit]

Two years ago, we did not have categories. Now almost every article and stub is categorized in some way.

Categories Count % Plot
0 18 6.5 *
1 73 26.5 *****
2 94 34.2 *******
3 39 14.2 ***
4 26 9.5 **
5 11 4 *
6 2 .7
7 3 1.1
8 4 1.5
9 1 .4
10 3 1.1
14 1 .4
Total 275

Formulae

[edit]

A rare sighting.

0 Formulae 273 99.3% ********************
1 Formula 1 .4%
3 Formulae 1 .4%

Maps

[edit]

Not very common, but then again, not every article should have one. Still, many place articles do not have a map. I counted 29 entries on places, and 13 maps...

0 Maps 262
1 Map 13

Pictures and Illustrations

[edit]

Most articles still come without illustration. I have not come across any animations or videos, something other encyclopaedias brag with... I know they exist, but the fact that they do not show in this assessment suggests that there are not many of them.

Frequency # % Plot
0 216 78.5 ****************
1 39 14.2 **
2 12 4.4 *
3 4 1.5
4 1 .4
6 1 .4
9 1 .4
20 1 .4
Total 275 100.0

References

[edit]

At least we have now started with references, but they are largely absent. This not only compromises the verifiability of entries, but is probably a sign of rampant plagiarism (rephrasing something does not mean the source should be attributed).

Frequency # % Plot
0 254 92.4 ******************
1 10 3.6 *
2 5 1.8
3 1 .4
4 4 1.5
5 1 .4
Total 275 100.0

Tables

[edit]

As with illustrations, not very common. There is of course the question whether tables should be used in some entries at all, because they are not always a useful way to summarize information.

0 Tables 227 82.5% ****************
1 Table 44 16% ***
2 Tables 4 1.5%

Length

[edit]

Again, like two years ago, I have used a rather convenient way to measure the length of entries.

Frequency Count % Plot
3 lines or less 69 25.1 *****
less than 1 screen 77 28.0 ******
1 Screen 29 10.5 **
2 Screens 51 18.5 ****
3 Screens 26 9.5 **
Longer 23 8.4 **

Britannica comparison

[edit]

As last time, I have checked whether the entry was also in Britannica. Last time I used the 2002 DVD version of Britannica; this time I used search.eb.com, so in Britannica also means in the 'Britannica Student Encyclopaedia. Not in Britannica means that there were no matches, In Britannica means that there is an article with the same or an equivalent title, Within Wider Britannica Article means that the topic is treated within a Britannica article that covers a larger topic. This is possibly my pet peeve: many stubs have little potential to grow because they would better be dealt with in a more general article (see WQA 1).

Not in Britannica 166 60.4% ************
In Britannica 32 11.6% **
Within Wider Britannica Article 57 20.7% ****

Areas covered

[edit]

Here are two tables on the areas the entries cover. First I included a category for persons, then I split this category to the others. So, in the first table, a politician would be counted as person, in the second she or he would be found under politics. The areas are to a large extent influenced by the articles, and do not follow an existing categorization. This must be borne in mind when considering the systematic bias of the Pedia.

Including persons

[edit]
Area Count %
Animal 6 2.2
Architecture 4 1.5
Company 3 1.1
Culture 4 1.5
Disambiguation 20 7.3
Geography 10 3.6
History 6 2.2
Language 2 .7
Law 5 1.8
Leisure 12 4.4
Literature 14 5.1
Media 12 4.4
Music 14 5.1
Nobility 6 2.2
Organization 9 3.3
Person 55 20.0
Place 30 10.9
Plant 2 .7
Internet 4 1.5
Religion 4 1.5
Science 13 4.7
Sports 10 3.6
Standard 2 .7
Technology 4 1.5
Misc. 5 1.8
Transport 7 2.5
Other 6 2.2
Warfare 6 2.2
Total 275 100.0

Persons assigned to other areas

[edit]
Area Count %
Animal 5 1.8
Art 15 5.5
Company 3 1.1
Culture 6 2.2
Disambiguation 20 7.3
Fiction 6 2.2
Geography 10 3.6
History 6 2.2
Language 2 .7
Law 5 1.8
Leisure 12 4.4
Literature 12 4.4
Media 15 5.5
Music 22 8.0
Nobility 6 2.2
Organization 9 3.3
Place 29 10.5
Plant 2 .7
Internet 4 1.5
Politics 10 3.6
Religion 9 3.3
Science 17 6.2
Sports 18 6.5
Standard 2 .7
Technology 4 1.5
Misc. 5 1.8
Transport 7 2.5
Other 6 2.2
Warfare 8 2.9
Total 275 100.0

Predicting what is an article

[edit]

The following table is the result of a regression analysis, trying to predict what makes an article (as opposed to a stub, fragment, or the like). Hits in Google and Scirus are insignificant predictors, meaning that some articles have many Google hits, others just a few. If the topic occurs in Britannica, it is 1.5 times as likely to be an article as a topic that does not. This, however, is not statistically significant once the number of incoming links is considered. The number of links to an entry is the single most powerful predictor whether an entry will be an article or anything less: every incoming link increases the chance of being an article by 8%.

Sig. Exp(B)
Is in EB .298 1.418
Google Hits .936 1.000
Scirus Hits .768 1.000
Scirus Journal Hits .446 1.000
Links In .000 1.078
Constant .000 .171

I have also run the regression with splitting the different kinds of Britannica entries (equivalent or within other article). Again, the number of Google hits is irrelevant. The number of incoming links is a very good predictor. Moreover, we can see why the distinction between the kinds of Britannica entry is important: a Wikipedia entry with an equivalent article in Britannica is 2.3 times as likely to be an article than an entry without. Wikipedia entries with an article in Britannica that covers a wider topic do not fare significantly better than entries that have no entry in Britannica at all. I take this as a sign that we have too many entries that have no potential to grow...

Sig. Exp(B)
1000 Google Hits .734 1.000
EB Equivalent .102 2.305
EB within Larger .605 1.216
Links In .000 1.076
Constant .000 .175

Predicting what is a stub

[edit]

Here are the results of a regression analysis that predicts whether an entry is a stub as opposed to anything else. We find the opposite here. Again, the number of hits in Google and Scirus are insignificant. The number of incoming links is significant: For every extra incoming link, the entry is 8% less likely to be a stub. If there is an entry in Britannica, the chances of being a stub drop by 34.7%.

Sig. Exp(B)
Is in EB .189 .653
Google Hits .792 1.000
Scirus Hits .774 1.000
Scirus Journal Hits .979 1.000
Links In .000 .912
Constant .000 2.253

Predicting article length

[edit]

This table summarizes the prediction of article length. All these predictors are statistically significant (.1 level), with the exception of the number of hits in Scirus. An entry which can also be found in Britannica is expected to be about half a screen larger than one that cannot. The effects of hits in the search engines are significant but very small. It takes 1 million Google hits to increase the article length by half a screen (0.000000492 screens for every Google hit); or about 5000 Scirus Journal hits for the same effect (0.0000935 screens for every Scirus Journal hit).

B Sig.
(Constant) 1.062 .000
Is in EB .582 .002
Hits in Google 4.92E-007 .004
Hits in Scirus -2.02E-005 .117
Journal Hits in Scirus 9.35E-005 .092

Features in longer articles

[edit]

Longer articles tend to have more of the features measured. All correlations are positive and significant at the .01 level. The number of incoming links once again shows up as the strongest effect.

Number of Maps .172 **
Number of Pictures .172 **
Number of References .179 **
Number of Tables .236 **
Links to Entry .473 **

The data

[edit]

Feel free to make use of the data for your own analyses. Bear in mind the predictive limits outlined in the introduction: Data.