Wikipedia:Wikipedia is failing

Is Wikipedia succeeding in its aim of becoming a reputable, reliable reference work? Here are some illustrations of ways in which it is not fulfilling that aim.

File:Wikipedia feedback loop.jpg

Success is not guaranteed.

Assumptions

To assess the quality of Wikipedia's articles, some assumptions are necessary. Here it is assumed that:

• The criteria defined by the Wikipedia 1.0 editorial team at {{grading scheme}} accurately reflect the quality of the articles these ratings have been applied to.
• That articles which are not either FA or A-class fall below the standards that a reference work should demand of its content.
• That the sample of 300,000 articles assessed, with results listed at WP:1.0/I, is representative of the whole encyclopaedia.

Criteria which indicate substantial failings

Performance on core topics

Vital articles lists 1182 articles on topics that can be considered essential. These topics should have articles of the very highest quality - ideally a featured article. So do they? In fact, of those 1182, only 72 are featured articles. This means that 94% of the essential topics that should have excellent articles fall short of the standard, assuming that all vital articles that meet the FA criteria have been nominated for FA status.

Do they fall short by a long way? 131 are listed as good articles, which, according to Template:Grading scheme, means that 'other encyclopedias could do a better job'. Some editors have criticised the GA process as inconsistent and arbitrary, so the quality of those articles is further in doubt. 133 are listed as articles which are either stubs or have a cleanup tag. The rest, presumably, are B-class or start-class on the assessment scale; this indicates that many articles require substantial work before they will match or exceed the standards found in other encyclopaedias.

On current trends, how long will it take before all the Vital Articles are featured articles? On 1 January 2006, 41 of them were featured; by 1 January 2007, this had risen to 71. As FA promotion rates have remained approximately constant for well over a year it would be difficult to assume anything other than a constant rate of VAs becoming FAs. At this rate of approximately 30 a year it will take 37 years for all of the vital articles to reach the standards expected of them.

Performance on broader topics

There are about 1,300 featured articles. There are also about 1,700 good articles. However, there are currently 6,898,301 articles on Wikipedia. This means that slightly more than 99.8% of all the articles on Wikipedia have not yet been assessed as featured or good articles. In many cases this is because they are not considered well-written, verifiable, broad, or comprehensive in their coverage. The results of the largest-scale assessment of Wikipedia content, covering 18% of the total number of articles, can be found at WP:1.0/I. These results show that 0.7% of assessed articles are either FAs or A-class articles.

One useful, informal exercise for a reader is to critically read ten random articles. The numbers above suggest that on average, you'd expect to find one FA or A-class article in every 143 articles you looked at (based on WP:1.0/I), or every 762 (based on total numbers of FAs and A-class articles).

Maintenance of standards

Do articles which are judged to have reached the highest standards remain excellent for a long time, or do standards decline as well-meant but poor quality edits cause standards to fall over time? There are currently 340 former featured articles, so that more than 20% of all articles that have ever been featured are no longer featured.

Many editors observe that an FA that is not actively maintained inevitably declines; for an example see Ryanair, which attracts large numbers of highly biased edits which have wrecked a formerly excellent article. Sun's lead section was reduced to a few short sentences by an editor who either hadn't read or didn't understand the guidelines on what a lead section is supposed to be, and no-one has restored the previously existing summary. A whole section of Mauna Loa was removed by a vandal in November, and was not restored for a month. Generally, if the primary author of an FA does not take care of it, checking changes up to several times a day, it is likely to have its quality compromised by unnoticed vandalism or, far more damaging in the long term, well-intentioned but poor quality edits.

Some or many articles may lose featured article status because they do not meet current standards, rather than because they have declined in quality. Without case-by-case analysis it is impossible to say what proportion this is the case for. However, we can note that the featured article review process has not been as successful as would be ideal at encouraging featured articles to improve in line with rising standards.

Rate of quality article production

Many argue that Wikipedia is a work in progress and that, given time, all articles will reach very high standards. Unfortunately, this is not borne out by the rate at which articles are currently being judged to meet featured article criteria. About one article a day on average becomes featured; at this rate, it will take 4,380 years for all the currently existing articles to meet FA criteria. If the current approximately exponential growth rate of Wikipedia (which will see it double in size in about the next 500 days) continues, then on current trends there will never be a time when all articles have been promoted to featured article status.

Should we even expect all articles to meet the featured article criteria? A majority of people who commented on one earlier discussion felt that the featured article criteria do indeed define the standards that all articles need to meet.

Is WP:FA a bottleneck? The rate at which articles have been promoted has remained more or less constant for well over a year (see WP:GAS), while article creation rates have increased exponentially throughout that time. If the system prevents large numbers of quality articles from being recognised as such, then that indicates that some kind of reform of the system is necessary.

Special:Recentchanges provides evidence that the rate of addition of substantial encyclopaedic content is low. You may find it informative to look at the last 200 recent changes and count how many of them are directly building the encyclopedia. That means observing reasonably sound content being added (and not under a 'trivia' header) to an article that is not a borderline AFD candidate. (The info on bytes added/removed narrows the search quite quickly.) Typically this reveals less than ten substantive article-space change in 200. One such analysis can be found at User:Opabinia_regalis/Article_statistics.

The strength and size of the core community

One of the most important aspects of Wikipedia is what could be called its "core community," as distinguished from the "community-at-large." The distinction is that the community-at-large is composed of everyone who edits Wikipedia, while the core community is the small group of veteran editors who regularly watch policy pages, facilitate the administration of various aspects of the site (such as mediation, the help desk, the reference desk, the various noticeobards, etc..). Earlier in Wikipedia's history, there was a project called Esperanza designed to strengthen Wikipedia's core community. In mid-2007, Esperanza was disbanded after facing criticism. At first, other projects were started to fulfill the same role that Esperanza did. Several of these projects have since been abandoned.

Projects stemming from Esperanza which were abandoned:

• Reach out
• Stress Alerts

Projects stemming from Esperanza which are still ongoing:

• Admin coaching
• Tutorial drive
• Birthday Committee/Calendar
• Article Collaboration and Improvement Drive.

Questioning these criteria

Is it a bad idea to use Featured Article or Good Article status as criteria for judging the number of excellent articles in Wikipedia? It is possible that many or most articles that meet the featured article criteria or good article criteria have not been officially reviewed, because review is a time-intensive process that often suffers from a backlog of nominated articles. The Good Article process historically has had a much less rigorous promotion process than the featured article process, so some editors reject it as a measure of article quality. In addition, the editors who work on articles that could potentially pass either the "good" or "featured" criteria may decline to participate in those processes because they see them as bureaucratic, unpleasant, and predominantly run by non-experts in specialized topics.

If these processes do not succeed in recognizing quality content, then this may be a failure of Wikipedia to perform accurate self assessment rather than a failure to produce quality articles.

Food for thought

If Wikipedia just aimed to be a social site where people with similar interests could come together and write articles about anything they liked, it would certainly be succeeding. However, its stated aim is to be an encyclopedia, and not just that but an encyclopaedia of the highest quality. Six years of work has resulted in 3,000 articles of good or excellent quality, at which rate it will take many decades to produce the quantity of good or excellent articles found in traditional reference works. Over 1.6 million articles are mediocre to poor to appalling in quality.

Open questions

• Has the system failed to produce a quality reference work? If so, why?
• Is change necessary?
• If it is, then is radical change required, or just small adjustments to the current set-up?
• Does this matter, given that Wikipedia is one of the most popular websites in the world?
• Does popularity establish authenticity?
• What is Wikipedia really, and what do we want it to be?
• Are the statistical measures introduced here relevant to the conclusions drawn?
• Are Wikipedia's own criteria for success accurately reflected here?
• Are the Featured Article and Good Article designations useful for determining the number of quality articles in Wikipedia? If they are not, how can they be reformed?
• At what rate is the number of new user accounts increasing?
• Does the number of active users increase in the same way as new user accounts, or do significant numbers of editors leave the project?
• Could it help any to introduce one or more of the following:

1. a clearer vision and mission statement, prominently displayed?
2. better defined performance metrics for articles or edits?
3. voting, as popularity, or by "distinguished members" (opening a can of chicken/egg soup here)?
4. profiling authors/editors to identify promising candidates or repeat offenders so as to offer them voluntary coaching/mentoring (on private channels, not in public on discussion pages)?
5. offering references to alternative sites so as to channel creative energies of writers who repeatedly fail to meet Wikipedia criteria?
6. leaving it as it is because the last few questions lead to an unworkable data nightmare?

Responses to Alleged Rebuttals

Response to rebuttals

See the sister essay, Wikipedia:Wikipedia is succeeding

The sister essay to this article relies on an "absolute measure" of Wikipedia's success, which doesn't seem to be quite what Jimbo originally had in mind when he founded Wikipedia along with the Wikimedia foundation's original definition of "success." The definition of success appears to have shifted because several years ago, Jim idealistically and hubristically called Wikipedia a "compendium of all human knowledge," while his more recent statements have suggested that Wikipedia is unreliable except as a starting point and should not be used in college papers. This appears to be an argument that "Wikipedia is succeeding" by lowering the bar for what an encyclopedia actually is.

According to the article on encyclopedia, an encyclopedia is, "A comprehensive written compendium that contains information on all branches of knowledge." That definition hearkens back to Jim's earlier hubristic assertions. While Wikipedia is not reliable enough to be used in college papers, it is certainly arguable that Britannica is reliable enough to be used, precisely because of its high relative accuracy, not just its absolute accuracy in comparison to other encyclopedias.

As an analogy of the sister's essays argument: Let us assume there are two pies, Britannica pie and Wikipedia pie. Both are equally as delicious and nutritious. But then, the Wikipedia pie is sitting on top of a mountain of garbage that is growing and growing exponentially each day. Since Wikipedia is defined as the "core articles that are also in Britannica" (the delicious pie) but also the "mountains of garbage" created by WP:PAPER, it's absurd to claim that Wikipedia is succeeding, based on such special pleading. It is a fact that much of Wikipedia's articles rely on sources even lower than tertiary sources, although this usually only applies in articles outside of the core articles. Keeping this in mind, so long as WP:PAPER is upheld and "success" is going to measured by what articles on Wikipedia are also on Britannica, it is absurd to call Wikipedia an "encyclopedia," but rather, it is "a good encyclopedia, sitting atop a mountain of garbage."

Some key points about their flawed methodology:

• Correlation does not imply causation. The sister essay seems to totally ignore this and play on public ignorance of statistics and science by evangelizing about how quickly Wikipedia is growing and how many absolute articles of good quality are made without consider the actual relative, marginal, and continuous change.
• As with any statistical analysis, relying on "absolute scales," is misleading and relative scales are better.
• The quality of other encyclopedias should be regarded as sufficiently random and Wikipedia's accuracy of certain articles which also happen to be on certain self-selected encyclopedias is a biased sample. Naturally, as Wikipedia has been compared with Britannica so much and articles like List of 1974 Macropædia articles and List of 2007 Macropædia articles, of course Wikipedia would be comparable in accuracy, because those articles are spotlighted and the sources for such are most easily available. In other words, Wikipedia users could simply be doing the kind of "legal copying" through writing articles from scratch while using the same sources and information, the way Citizendium does to avoid violating Wikipedia's GFDL license.
• Most of the analysis was not carried out by neutral parties, but by Wikipedia editors themselves who have an agenda to consider.
• Their results have not been replicable by outside groups, only a few studies which themselves relied on similarly flawed methodology.

Most of all, in order to maturely and appropriately address this question, a common public misconception must be made explicit: Just because "a scientist did a study that proved it" doesn't necessarily mean it's true. The sister essay seems to rely heavily on this absurd notion, without actually examining the particular methodologies of the outside studies it invokes.

One of the absolute weakest rebuttals to this essay was on the issue of how long it would take for all of Wikipedia's articles to be featured. This essay argues that, at current speed it will take roughly 4,380 years for all currently existing articles to be featured. The rebuttal was:

Wikipedians are very patient. Furthermore, many of the articles currently not featured will most likely never be featured since there are many topics which meet Wikipedia notability criteria but do not have enough verifiable information about them to have as featured articles.

. Furthermore, many of the articles currently not featured will most likely never be featured since there are many topics which meet Wikipedia notability criteria but do not have enough verifiable information about them to have as featured articles." This is a laughable rebuttal that is so patently absurd it is difficult to come up with a serious logical objection, other than an invocation of the Fenyman algorithm.

This is not to appeal to ridicule, but similarly: When a cult member argues that there is Xenu or some such nonsense, how does one actually engage them logically? You don't. It is precisely the point that lots and lots of unencyclopedic articles are created every day. This assumption is implicit in their rebuttal, yet at the same time it is an assertion they reject. The claim "Wikipedia users are patient," is not established in their essay and it is pretty irrational to assume that most Wikipedia users are patient enough that they will wait four millenia for Wikipedia to even come close to Britannica's relative accuracy.

• Shortly, Wikipedia will be transforming itself into a magazine, which will contain news items, and featured articles for the month. It is hoped that this will draw attention to quality and increase the number of editors who are willing to finish articles.

As for German Wikipedia's success (or rather "smaller degree of failure"), this could likely be explained by a number of policy reforms which have helped to address Wikipedia's problem of anti-elitism. Kim Bruning has described German Wikipedia as an "Adminocracy," while he acknowledges that Dutch Wikipedia is crumbling under bureaucracy. English Wikipedia appears to fall somewhere in between the two and if German Wikipedia is truly so successful, policy proposals adopted by German Wikipedia ought to similarly be adopted by all other Wikipedias.

See also

• Criticism of Wikipedia
  • Eric Goldman
• Wikipedia:Wikipedia is not failing - this essay sets out specifically to rebut the arguments presented here.
• Wikipedia:Expert retention
• Wikipedia:Anti-elitism
• Wikipedia:100,000 feature-quality articles
• Wikipedia:Evaluating Wikipedia as an encyclopedia
• Wikipedia:Problems with Wikipedia
• User:Moreschi/The Plague

And finally...

If this essay makes you feel angry, stressed or miserable, the essay Wikipedia:Wikipedia may or may not be failing may or may not make you feel less or more so.