Wikipedia:Reliable sources

There is general agreement that Wikipedia articles should use reliable sources. This page is an attempt to provide guidance about how to identify reliable sources and reliable information.

If you can provide useful information to Wikipedia, please do so, whether or not you have a reliable source. But bear in mind that edits for which no credible references are provided may be deleted by any editor.

The best practices which are described here are ideals. Most articles will fall short of these ideals until one or more editors devote a significant amount of time and effort to fact-checking and reference-running. See Efforts to identify reliable sources at the bottom of this page for collaborative projects to do just that. In the meantime, readers can still benefit greatly from your contributions.

There are many ways in which factual errors can be introduced into reports. Different types of claims may require different methods for determining their accuracy. Keep in mind that some articles are about characterizing the various factions in a dispute. This means that you will be looking for reliable reports of other people's opinions.

Some definitions

Please note the following terms:

• A fact is an actual state of affairs, which can be an historical event, or a social or natural phenomenon. To say of a sentence or proposition that it is true is to say that it refers to a fact. (Although a fact is always true, by definition, not all claims of fact are true.)

• An opinion is a view that someone holds, the content of which may or may not be verifiable. However, the idea that a certain person or group of people holds a certain opinion is a verifiable fact.

• A primary source provides direct evidence for a certain state of affairs. This may mean that the source observes a state of affairs directly, or that they observe indirect evidence of it. In other words, a primary source is a source very close to the original state of affairs you are writing about. An example of primary-source material would be a photograph of a car accident taken by an eye witness, or a report from that eye witness. A trial transcript is also primary-source material.

• A secondary source summarizes one or more primary sources. A tertiary source summarizes one or more secondary sources.

Hearsay

The longer the chain of sources between the reader and the actual event, the more likely it is that mistakes and misrepresentations have crept into the report. The human brain is best at remembering the gist of what it has been told, but usually does not record the exact words. Different authors may also make different assumptions, which are not necessarily explicit and which may lead to an incorrect interpretation of the events being reported. This is why courts of law exclude hearsay. It is also why rumors are considered unreliable and contribute to the creation of some urban legends.

Whenever possible, Wikipedia editors are encouraged to follow the chain of reporting, or chain of evidence, back as far as possible.

Get close to the source

When reporting on objective facts, Wikipedia articles should cite primary and secondary sources whenever they exist. Tertiary sources like reputable encyclopedias (such as the Encyclopædia Britannica), as well as reputable specialized encyclopedias such as the New Grove and the Dictionary of National Biography, contain a wealth of reliable information. Older editions such as the 1911 Encyclopædia Britannica often have fuller articles than current editions on some subjects, though there is always the danger that the information is outdated.

If suspect sources have references, follow them. If there are no references, or if the references provided are insufficient, you may need to do additional research, or reconsider the reliability of the report.

When reporting that an opinion is held by a particular individual or group, the best citation will be to a direct quote. If there is text, audio, or video available of someone expressing the opinion directly, it is preferable to include or transcribe an excerpt (very much allowed under fair use).

Unattributed material

Wikipedians often report as facts things that they remember hearing about or reading somewhere, but they don't remember where, and they don't have any other corroborating information. It's important to seek sources to verify these types of reports, and if they cannot be verified, any editor may delete them.

It's always appropriate to ask other editors, "How do you know that?", or "Can you cite your source?" If they didn't have a particular source in mind when they wrote the material originally, someone will have to find a credible source. The burden of evidence lies with the editor who has made the edit in question.

Many popular misconceptions can be rooted out of the encyclopedia in this manner. Even when editors are remembering facts correctly, the details may be fuzzy, and it is enlightening and satisfying for all involved when more exact, reliable information is uncovered.

For advice about dealing with unattributed material when you find it, see Wikipedia:Verifiability, Wikipedia:No original research, and Wikipedia:Cite sources.

Evaluating primary sources

• Do they have an agenda or conflict of interest, strong views, or other bias which may color their report? Remember that conflicts of interest are not always explicitly exposed and bias is not always self-evident.
• Were they actually there? Be careful to distinguish between descriptions of events by eyewitnesses and by commentators. The former are primary sources; the latter secondary.
• Find out what other people say about your sources.
• Have they reported other facts reliably, including on different subjects? Cross-check with what you already know.
• Are they available for other editors to check?

See Wikipedia:No original research and Wikipedia:Cite sources for more details.

Check multiple independent sources

It is very important to cross-check primary sources against each other. This will help detect biases and errors.

Psychological experiments have shown that memory and perception are not as reliable as we would like them to be. In one experiment, subjects were shown playing cards with some anomalies. Subjects could usually identify normal cards correctly if even they were displayed for a very short amount of time. But when briefly flashed a black four of hearts, for example, most subjects would, without apparent hesitation or puzzlement, incorrectly identify it as either the four of hearts or the four of spades. Subject only became aware of the anomalous cards with longer exposures. Subjects who were aware that these strange cards were mixed in with normal cards were also much better at identifying them. [#References 1]

Recent scientific experiments have begun to explain how the brain can remember imagined events as if they were real. [#References 2]

Police, judges, and trial lawyers are familiar with the phenomenon that several different people witnessing the same event remember it differently, sometimes including crucial details.

Most people also know that the older a memory is, the less reliable it may be. Recent studies have shown that this may be because memories are overwritten each time we access them. [#References 3]

Because conscious and unconscious biases are not always self-evident, you shouldn't necessarily be satisfied with a single primary source. Find another one and cross-check. If multiple independent sources agree and they have either no strong reason to be biased or their biases are at cross purposes, then you have a reliable account.

What is an independent primary source?

Independent primary sources:

• Each had direct personal experiences which they are recounting
• Have not discussed their experiences with each other, which could contaminate their memories of events
• Do not have a common influence which could taint their stories in the same way.

Editors may only use information that has been published in some form already by a credible publisher, so that we can offer that publication as a citation.

Evaluating secondary sources

• Are they actually a secondary source (e.g., have they mainly derived their information from primary sources)?
• Have they used multiple independent primary sources?
• Do they have an agenda or conflict of interest, strong views, or other bias which may color their report? Remember that conflicts of interest are not always explicitly exposed and bias is not always self-evident.
• Find out what other people say about your sources.
• Have they reported other facts reliably, including on different subjects? Cross-check with what you already know.
• Are they available to other editors to check? If not, inclusion is probably not appropriate, unless the information is not available elsewhere.

Check multiple independent sources

Even given the same primary sources, different analysts may come to different conclusions about the facts being reported. In practice, many secondary sources find and use different primary sources in the course of their research.

Conscious biases, unconscious biases, and errors are not always self-evident. The best way to expose them is to cross-check with an independent source.

What is an independent secondary source?

Independent secondary sources:

• Have unrelated entities of editorial oversight. This means that they have different employers, different editors, and possibly different publishers.
• Have not collaborated their efforts.
• Have taken their own look at the available primary sources and used their own judgment in evaluating them.

Exceptional claims require exceptional evidence

Certain "red flags" should prompt editors to closely and skeptically examine the sources for a given claim.

• Reports of a statement by someone that seems out of character, embarrassing, controversial, or against an interest they had previously defended.
• Surprising or important facts which are not widely known.
• Surprising or important recent events which have not been reported by reputable news media.
• Claims which are not supported or which are contradicted by the prevailing view in the scientific community.
• Claims which strongly support one or another party in an ongoing dispute (see e.g. Wikipedia:List of controversial issues)

Domain-specific advice

Personal opinions

Statements of personal opinion can easily be misheard, misunderstood, misinterpreted, twisted, and sensationalized.

When evaluating reports of such statements, consider whether or not the statement aligns with the views of the speaker as expressed directly by them or as reported in other sources. If you are not familiar with the public views of the person in question, dig around and find out more about them. It may not be apparent that your source is biased or unreliable unless you have some background information to check it against.

Opinions on some subjects are more easily misinterpreted than others. Use your best judgment.

History

Historical events are difficult to verify. We must rely on people's memories, recorded accounts, and physical evidence to reconstruct it.

Evidence and factual accounts contemporary with the events are valuable because there are fewer steps separating them from reality. On the other hand, their authors are more likely to have been involved in the events, and therefore to have a particular point of view. Indeed, the evaluation and judicious use of primary sources is a major part of the craft of history.

Summaries and overviews of history require interpretation and analysis, finding patterns and attributing causes. Sometimes later historical analyses of this kind are more reliable, because the passage of time allows more scholarly debate, more reflection, and decreases the likelihood that the historian was personally involved in or attached to the events that he or she is analyzing.

Science and medicine

Cite peer-reviewed scientific journals and check community consensus

Scientific journals are the best place to find primary source articles about scientific experiments, including medical studies. The best scientific journals are peer-reviewed, which means that independent experts in the field are asked to (usually anonymously) review articles before they are published. This usually results in corrections and improvement, sometimes substantial. Many articles are excluded from peer-reviewed journals because they report questionable (or perhaps merely unimportant, in the opinion of the editors) results.

• The fact that a statement is published in a peer-viewed journal does not make it true.

Even a well-designed experiment or study can produce flawed results. (For example, see the Retracted article on neurotoxicity of ecstasy published in the very prestigious journal Science.) The second part of peer review occurs after publication. Colleagues in the field will read the journal article and discuss it in various forums, including other journals in the same field and often later articles in the same journal. They may find self-evident flaws in the procedure used just by reading the article and applying their experience, or it may take a long process of trying to reproduce the results by similar or completely different means for the scientific community to determine that the original results were corrupted by some undetermined methodological problem, or to rigorously confirm the original result. The most reliable source for scientific information is the prevailing "scientific consensus".

Determining the scientific consensus could be done with a survey of experts in the field, but it can also be accomplished by following the state of discussions in respected journals.

At some points in time, there is no one prevailing view because the available evidence does not yet point to a single answer. Because Wikipedia not only aims to be accurate, but also useful, it generally tries to explain the theories and empirical justification for each school of thought, with reference to published sources. Editors should not, however, create arguments themselves in favor of, or against, any particular theory or position. See Wikipedia:No original research, which is policy.

• Just because something is not an accepted scientific fact, as determined by the prevailing scientific consensus, does not mean that it should not be reported and referenced in Wikipedia.

However, although minority views are welcome in Wikipedia, the views of tiny minorities need not be reported. See Wikipedia:Neutral Point of View. Minority views should be reported as that, and should not be given the same amount of space in an article as the majority view.

Simply make readers aware of any uncertainty or controversy. A well-referenced article will point to specific journal articles or specific theories proposed by specific researchers, rather than: "Some say that ... (vague, unattributed theory), but others believe ... (vague, unattributed theory)."

Which science journals are reputable?

A good way to determine which journals are held in high esteem without polling a bunch of scientists is to look at Impact Factor ratings, which track how many times a given journal is cited by articles in other publications.

Keep in mind that even a reputable journal may occasionally post a retraction of an experimental result. Articles may be selected on the grounds that they are interesting or highly promising, not merely because they seem reliable.

Evaluating experiments and studies

There are certain techniques that scientists use to prevent results from being contaminated by certain kinds of common errors, and to help others replicate results.

• Experimental control
  • Placebo controls
  • Ensuring demographic information aligns with the general population to check that the sample is sufficiently random
  • Double-blind medical studies
• Present a high degree of detail about the design and implementation of the experiment; don't just present the results.
• Make raw data available; don't just present conclusions.

Beware false authority

Would you trust a plumber to reliably and painlessly fill your cavities? We hope not. Likewise, you should probably not trust someone who has a Ph.D. in plant biology to tell you about quantum mechanics.

Just as actors in TV commercials don white lab coats to make viewers think they are serious scientists, people with graduate degrees in one field will flaunt it in an attempt to fool people into believing they are experts in a completely different field, or in all fields. Watch out for this and other false claims of authority.

You can be slightly less skeptical of experts who have degrees in the field they are discussing. The more reputable ones are affiliated with academic institutions. The most reputable have written textbooks in their field for the undergraduate level or higher: these authors can be expected to have a broad, authoritative grasp of their subject.

Statistics

Statistical information is easily and often misinterpreted by the public, by journalists, and by scientists. It should be checked and explained with the utmost care, with reference to published sources.

See Misuse of statistics, Opinion poll, and Statistical survey for common errors and abuses.

Using online sources

Don't drop your guard

Evaluate the reliability of online sources just as you would print or other more traditional sources. Neither online nor print sources deserve an automatic assumption of reliability by virtue of the medium they are printed in. All reports must be evaluated according to the processes and people that created them. Publications with teams of fact-checkers, reporters, editors, lawyers, and managers, like the New York Times or The Times of London, are likely to be reliable, and are regarded as reputable sources for the purposes of Wikipedia. At the other end of the reliability scale lie personal websites and weblogs, which are not acceptable as sources. Partisan political and religious sources should usually be avoided for the same reason, or treated with caution. Very extreme political websites should never be used as sources for Wikipedia except in articles about that organization, where its views may, of course, be referred to.

Many websites are created by unknown individuals who have no one checking their work. They may be uninformed, misled, pushing an agenda, sloppy, relying on rumor and suspicion, or insane; or they may be intelligent, careful people sharing their knowledge with the world. Visiting a stranger's personal web page is often the online equivalent of reading an unattributed flyer on a lamp post, and should be treated accordingly — with significant skepticism.

Great for easy access

Full-text online sources are preferable to offline sources if they are of similar quality and reliability because they can be more easily accessed by other editors who wish to check references, and by readers who simply want more information.

If you find a print source that is out of copyright or that is available on compatible licensing terms, add it to Wikisource and link to it there (in addition to the normal scholarly citation). Many significant out-of-copyright books have already been put online by other projects.

Don't be lazy

Until more authors publish online and more material is uploaded, some of the most reliable and informative sources are still in print. If you can't find a good source on Google, get to your local library or bookstore. They exist for a reason. You'll be amazed what you can learn there.

Efforts to identify reliable sources

• Wikipedia:WikiProject Fact and Reference Check
• Wikipedia:Forum for Encyclopedic Standards
• Wikipedia:Approval mechanism

See also

• Wikipedia:Neutral Point of View
• Wikipedia:No original research
• Wikipedia:Cite sources
• Wikipedia:Verifiability
• Wikipedia:Check your facts

References

1. The playing card experiment is described by Thomas Kuhn in The Structure of Scientific Revolutions (1962), p. 62-64. He cites the following article: J.S. Bruner and Leo Postman, "On the Perception of Incongruity: A Paradigm," Journal of Personality, XVIII (1949), 206-23. Following the advice of this page, the original source should be checked to see if the summary of Kuhn's summary is accurate.
2. False memories based on imagined events:

"Biological Basis for False Memories Revealed" by Michelle Trudeau. All Things Considered 23 Oct 2004. [1]

"Making False Memories." Talk of the Nation Science Friday. 4 Feb 2005. [2]

1. On overwriting memories each time we access them:

(Currently locating sources.)