Wikipedia talk:Verifiability

WP:PAYWALL and freely accessible sources

Earlier today, I made this edit to the page, but was reverted six minutes later. What I wished to add to the section on accessibility of sources was "However, in cases where an equally reliable and more easily accessible source can be located, it should be used in preference or in addition to a source that is more difficult to access." The intent of this was to promote the use of peer-reviewed open access journals and the like, so that it would be easier for people wishing to check references and research further to do so. I am not sure why that is a bad idea... NW (Talk) 22:48, 16 August 2010 (UTC)

The reverter didn't say it was a bad idea, they said that it needed to be discussed first. Sincerely, North8000 (talk) 22:57, 16 August 2010 (UTC)

(edit conflict) I suppose I reverted the edit for the same reason you are in favor of it: it is not our place to promote (or not promote) open-access journals, or indeed any source over any other source. Whatever source the editor has at hand should suffice, and I don't think our policies should show favoritism beyond the minimal requirements of verifiability. Of course, if an editor wishes to refer to open source journals that is fine, so long as they are reliable sources, but that is different from saying that everyone should prefer the use of such journals. RJC ^Talk_Contribs 22:59, 16 August 2010 (UTC)

The vast majority of people reading Wikipedia will be doing so from home, office, or mobile internet connections. A very small subset will be reading from academic and industry institutions and corporations that would have access to journals that one would have to pay for. Clearly, something like BMJ is just as reliable as Journal of the American Medical Association. The former is freely accessible and the latter is pay-to-view. Why should we not encourage the use of the former over the latter, or at the very least, both instead of just the latter? NW (Talk) 23:15, 16 August 2010 (UTC)

Personally I don't approve of the proposed addition. The ease of access to or availability of a source has never, and should never, be a factor in verifiability. I know it is not the intent of the change... but people would use it to argue that we should prefer on-line sources over dead tree sources (and I strongly disagree with that idea). Blueboar (talk) 23:21, 16 August 2010 (UTC)

I would definitely strongly oppose something like that. Could you think of anyway to wordsmith my addition to bring it more in line with what I actually meant? (see my BMJ/JAMA example above) NW (Talk) 23:24, 16 August 2010 (UTC)

It may be true that most readers will not have access to the best sources, but I don't know that is a reason to prefer one set of sources over another. Verifiability is there to make sure we aren't just making stuff up. The proposed addition would augment that mission, I think. RJC ^Talk_Contribs 00:51, 17 August 2010 (UTC)

I'm sorry, could you clarify? I'm not sure if I understood you properly. NW (Talk) 02:02, 17 August 2010 (UTC)

I think BMJ vs. Journal of the American Medical Association if one finds an article in one that provides useful information for an article, it is unlikely one will find an article in the other that is equally suitable. A better example would be a modern paper printing of a 19th century English novel vs. the Project Gutenberg edition of the same novel. In any case, I think the editors are frugal enough that there will be little problem with using sources that are more expensive than necessary, and I'd hate to put in anything that can be used as an excuse to suppress material from high quality journals. Jc3s5h (talk) 02:28, 17 August 2010 (UTC)

I'm sorry. I was trying to say that the fact that the majority of Wikipedia's readers do not access it from a library where they could find the traditional journals is not a reason to change the policy. The purpose of verifiability is to ensure that our content is high quality, even accurate. I would say that anything that does not contribute primarily to that goal should not be a part of policy. Otherwise, we get into issues of mission creep. So, a preference for open-access journals does not contribute to the core mission of verifiability, however much consensus there might be among Wikipedia editors that they would like free access to the latest research. I don't think we should have a preference for or against open-access journals as a part of our policy on what it takes to show that we aren't just making the encyclopedia up as we go along. RJC ^Talk_Contribs 14:07, 17 August 2010 (UTC)

Actually, emphasizing paywalled sources can have a devastating effect on quality in that they are quite difficult to verify. Thus someone can cite a paywalled source as saying something when the source does not say it at all, and the chances that it gets verified are slim to none. See Wikipedia:Requests_for_comment/Jagged_85 for an example. On the other hand, if someone uses an open-access/self-archived source, it may be a little lower-quality but it has still been peer-reviewed and we can easily check that it says what it is purported to say. Further, the readers benefit enormously from having much more detail available at the click of a link. I can understand the concern with the language, but ultimately I support NuclearWarfare's edit to the page and personally nearly always search for open-access sources as a starting point. II | (t - c) 14:16, 17 August 2010 (UTC)

The purpose of verifiability is to ensure that our content is high quality, even accurate. I would say that anything that does not contribute primarily to that goal should not be a part of policy. - I absolutely agree with this, but you seem to be overlooking that the free availability of a source very directly contributes to that goal. The point of citing sources is to enable others to check statements, and the easier it is made for them, the better the factual accuracy will become. To put it another way, it is already a well-known strategy among hoaxers to cite publications that are reputable but are hard to access.

Regards, HaeB (talk) 14:29, 17 August 2010 (UTC)

The problem with hoaxes is an interesting one, although I haven't seen it. In any case, it is one thing to build an case around an article in a 1914 journal that only ever published 3 volumes and is currently held only in Winnipeg. It is another to require that a journal have free online access. What precisely is the change intended to accomplish? Are we going to replace non-open-access citations with open-access ones? That would just be silly and wouldn't contribute to solving the hoax problem. Are we going to augment existing citations with open-access ones? That is already possible under existing policies, although it would be more the realm of a WikiProject than a policy change. Are we going to demand that, before someone adds a citation, they search for an open-access alternative? That sounds like a make-work project and certainly not something that should be built into policy (concurring with WP:DIG, WP:KISS, and WP:KUDZU). So, while I agree that the core mission of verifiability could be enhanced if facts were easily verifiable, a stated preference for open-access journals does not seem to be a good way to do so. Haven't we already discussed whether sources must be online, or whether we should prefer online sources, and come out against that? This seems even narrower than that case. RJC ^Talk_Contribs 14:45, 17 August 2010 (UTC)

Exactly... we want the most reliable sources for any statement. Ease of access has nothing to do with reliability. Sometimes the most reliable source will be easily accessible... sometimes the most reliable source will not be easily accessible. We still want the most reliable source. Blueboar (talk) 14:53, 17 August 2010 (UTC)

One example of this hoaxing strategy (the one I remembered offhand, there are many more):

I appended a footnote to the section calling Handel a "Nazi" with a reference to an article in the Oxford Times, a reliable source, to be sure, but one of which there is no digital archive anywhere ... (WebCite). What could he [the Wikipedian trying to fact-check the fake] do now? It was referenced to a "reliable source" and therefore untouchable, never mind that no such article had ever been published.

This lead to the hoax article being featured on the Main Page for six hours, see Wikipedia:Wikipedia Signpost/2010-03-08/News and notes.

As for your objections, they are mostly valid in principle, but seem to be addressed in NuclearWarfare's suggested wording, esepecially by the "equally reliable" precondition. Of course one wouldn't want to discourage offline or paywalled sources altogether (i.e. when there is no such alternative), I have cited many such sources myself. Regards, HaeB (talk) 15:15, 17 August 2010 (UTC)

If anything, I would say that "equally reliable" precondition makes things less problematic. How would my objections not refer to any statement of preference, not matter how worded? RJC ^Talk_Contribs 15:23, 17 August 2010 (UTC)

Yes, I too would you agree that the phrase inclusion of the phrase "equally reliable" would make my addition not-problematic. Is that what you meant? NW (Talk) 15:36, 17 August 2010 (UTC)

No, I think the statement of preference is itself problematic. HaeB suggested that it was only the "equally reliable" phrase that made it problematic. I think its absence would make the statement of preference more problematic, but even with it the statement still seems to run into the problems with any statement of preference (either non-open-access sources must be replaced, or they must be there in tandem, or editors should do additional work, none of which seem appropriate policy changes). RJC ^Talk_Contribs 15:40, 17 August 2010 (UTC)

I think this is a bad idea. This page should not be advocating selection of sources based on price. The concept would make a fine essay, though. WhatamIdoing (talk) 16:30, 17 August 2010 (UTC)

reboot

Rebooting this then, as I see valid comments made above. What about an addition like this:

However, in cases where the best source will be difficult for the layperson to obtain, if possible, editors should (but are by no means required to) attempt to add a second reliable yet more accessible source to the article in addition to the best source.

Not really the best of wording, I think you can get my intention? NW (Talk) 20:28, 17 August 2010 (UTC)

It still sounds like an essay or WikiProject, not a policy or guideline. RJC ^Talk_Contribs 20:58, 17 August 2010 (UTC)

Policies merely encourage things all the time, see WP:NONENG and all of Wikipedia:Conflict of interest. NW (Talk) 21:07, 17 August 2010 (UTC)

Note also that the content guideline Wikipedia:Identifying reliable sources (medicine-related articles) says "When all else is equal, it is better to cite a source whose full text is freely readable, so that your readers can follow the link to the source." NW (Talk) 18:23, 18 August 2010 (UTC)

You also have to consider the longevity of the source. Those paywall journals, online or in-print, are likely to be around forever. A free web-only source can disappear, and many have. I prefer a dead tree source whenever possible. -- SWTPC6800 (talk) 00:46, 19 August 2010 (UTC)

That is why one would cite both, and use webcitation if there is even a chance that it would disappear... NW (Talk) 03:41, 19 August 2010 (UTC)

My personal opinion is that such paywall sources should only be used if someone has actually looked at the paper being referenced in full. (Sometimes it's possible to find a copy of the PDF of such a source published unofficially on the internet.) Very often we end up relying only on the abstracts of the articles which can be a little misleading at times.

To look at all such sources will sound expensive. However WP has allot of contributors who have academic positions as students, or faculty at universities. Many universities have institutional subscriptions to thousands of journals. So we should try to enlist such Wikipedians to look these sources up for those who cannot. I know that sounds like allot of work. Did anyone say writing an encyclopedia would be a walk in the park? --Hfarmer (talk) 13:53, 19 August 2010 (UTC)

We should try to make it as much of a walk in the park as possible, lest "the encyclopedia anyone can edit" in practice mean "the encyclopedia anyone without a job can edit." At the very least, we shouldn't make it more difficult. RJC ^Talk_Contribs 14:06, 19 August 2010 (UTC)

"Paywall sources should only be used if someone has actually looked at the paper being referenced in full." Game, set and match to Hfarmer. Short Brigade Harvester Boris (talk) 15:15, 19 August 2010 (UTC)

We already have that service: Wikipedia:WikiProject Resource Exchange. Additionally, most of the larger WikiProjects have a few people who have access to an unusual number of sources in the relevant area. WhatamIdoing (talk) 20:41, 20 August 2010 (UTC)

I have access to most paywalled sources, but the average reader will not, nor will they have any idea about that particular WikiProject. That's who my addition was aimed at. NW (Talk) 23:52, 20 August 2010 (UTC)

"Those paywall journals, online or in-print, are likely to be around forever" - for online journals, this is unfounded speculation. There is good reason to assume that quite the opposite is true. While the library system ensures that old print journals remain available when their publisher goes out of business, there isn't much experience yet with what happens when the publisher of a paywalled online journal goes out of business. It is very well possible that old journals will become unavailable when the company's servers are switched off. Open access journals, on the other hand, benefit from the possibility of archival by third parties (many are already doing so, e.g. the Internet Archive). Regards, HaeB (talk) 04:19, 26 August 2010 (UTC)

The recent changes to WP:PAYWALL are encouraging people to include sources that they have not actually used. IMO this is borderline dishonest: editors need to cite their real sources, not "pretend" sources that they found later.

Wikipedia is an encyclopedia, not a bibliography or cheat-sheet for college students who aren't allowed to cite Wikipedia on their papers. We need to name the actual sources that we actually used. If the fact is easily verifiable through other sources, then that's great -- but our readers are smart enough to ask their favorite web search engines for alternative sources; we don't need to spoonfeed them. WhatamIdoing (talk) 20:39, 20 August 2010 (UTC)

Even "if it can be verified that both support the claim", there's still no plausible reason for demanding that editors produce twice as many citations as are actually necessary. Either PAYWALLed sources are sufficient and appropriate, or they're not. This idea that editors who are using high-quality sources need to spam a bunch of free (as in "free beer") sources into an already-well-cited article is not working for me. WhatamIdoing (talk) 21:26, 20 August 2010 (UTC)

Asking people to perform a simple courtesy if they can ("should", not "must") does not equal demand. Our purpose is to provide information to readers, including follow-up citations should they wish to look information up. If the vast majority of them cannot access, for example, an article in Nature, then we can at least do them the courtesy of linking them to an open access journal that they can read. If the book used to support a quotation by a historical figure is out-of-print and the same information might exist on a university website, then why shouldn't we encourage people to link that as well? NW (Talk) 23:51, 20 August 2010 (UTC)

I think paywall and offline sources are generally fine, but there are occasions where they're problematic. I ran into one bad-faith editor who deliberately used hard-to-check sources as a way to sneak bogus information into an article - he'd cite things like "Interview in Playboy Magazine, 1987", or old copies of small-circulation newspapers that don't have an online archive, or records in the archives of a specific court. On the occasions where we were able to check his sources, we almost invariably found they didn't support his edits; in some cases they didn't even exist as cited. He made a lot of use of socks and IP edits, so banning didn't solve the problem.

In that situation, I don't think the usual approach works - it puts us in a position where one malicious editor can spend a couple of minutes faking a source, and we have to spend much longer checking it before we can remove the 'cited' rubbish (from a BLP, at that). Context matters. --GenericBob (talk) 00:58, 21 August 2010 (UTC)

That is a problem, but I don't know that this addition is a way to resolve that. The person doing that would still claim, after all, that they don't have a good easily-available source, which would be fine even under the new wording. Again, since I do not see what good the proposed addition can do if it is reasonable, I oppose adding anything to its effect (since for it to have an effect, it must be an unreasonable request for additional busiwork, not citing the source they actually used, etc.). RJC ^Talk_Contribs 02:05, 21 August 2010 (UTC)

I agree with NW that some words to this effect would be helpful, because editors are increasingly using material that can only be accessed (easily) via a university database. So I would support wording along the lines of: "When source material lies behind a paywall, editors are encouraged, though not required, to supply a second, equivalent source that is more accessible." I can't see the harm in putting it in the form of a gentle reminder. SlimVirgin ^{talk|contribs} 05:37, 21 August 2010 (UTC)

I also support NW's new wording -- it seems very sensible to me to include a suggestion to keep the majority of our readers (who don't have access to academic journals) in mind, and provide free alternatives in addition to whatever non-free sources we might have, whenever possible. This will make verifying information easier, and will increase people's confidence in the correctness of the articles in question. Even if you wish to ignore the (bulletproof) argument that non-free sources are much less likely to be verified and are likely to be abused, there is still no harm that would come from including NW's addition. If people want to ignore the suggestion, they can. -- Jrtayloriv (talk) 07:01, 21 August 2010 (UTC)

Why require double citations?

If the community has a goal of providing free-beer sources to readers, then the community should have enough courage to come out and say that: i.e., "If you have a choice between a (at least minimally acceptable) free-beer-online source and an excellent offline source or a high-quality source behind a paywall, then you should always choose the free-beer-online source for the convenience of readers and future editors".

And if we don't happen to believe this, then we shouldn't be pushing double citations, and should instead let readers and future editors find their own free-beer-online sources. (After all, if it's always so quick and easy for the original editor to search for and cite two perfectly good sources for every statement in an article previously supported by a PAYWALLed source, then presumably future editors will be smart enough and have plenty of time to find just the second source themselves.)

I don't think that you're all grasping reality here: Double citations cost us time and energy, and the only people who will follow the directions are the ones whose time could be better spent on more productive activities. Maybe the thing to do is this: Go pick a long, well-written article in some technical field (which can be presumed to have lots of non-free-beer sources). I'll suggest Schizophrenia, if you can't find one on your own. Identify and double-cite every single one of the claims to make it be a stellar example of what you think is best practice. Then come back and tell us how long that took you. WhatamIdoing (talk) 18:50, 21 August 2010 (UTC)

While I support your strong defense of ensuring we use high quality sources, you don't appear to be addressing the modifications suggested here. The suggestions are not REQUIRING easily reached resources (free doesn't always mean poor), only SUGGESTING they be provided if the editor chooses and the editor finds EQUIVALENT, easily reached resources. --John (User:Jwy/talk) 19:01, 21 August 2010 (UTC)

I strongly oppose any requirements here, and have serious concerns as to "suggesting". A reliable source is a reliable source, whether it is easy to access or not. We should not even hint that free-online-easy to access is in some way "better", because that simply isn't true. My problem is that by suggesting it, we imply a preference that does not exist. Blueboar (talk) 19:16, 21 August 2010 (UTC)

That is an argument I struggle with. It is true accessibility does not influence the reliability of a source and should be ignored in assessing reliability. But more people are available to verify the information if it is in a more accessible place. If the same article is available both behind a paywall and free, Wikipedia would be better served if the free version is referenced. But trying to say that in these guidelines without diluting the important message of using the best sources may prove impossible. --John (User:Jwy/talk) 20:19, 21 August 2010 (UTC)

Why should we not "hint that free-online-easy to access is in some way "better"? It is better in one way -- namely that any editor can quickly and easily verify information from them, whereas with non-free sources, most people will not be able to verify things easily, if at all. This is not to imply that free sources are generally "superior" or "better". But as far as accessibility and ensuring accuracy of our articles through verifiability, free sources are more helpful to the large majority of users. Nobody is talking about forcing people to choose free articles over non-free, or forcing them to find free articles. They are only suggesting that we make a suggestion that if you feel like it it might be helpful (not superior) to find high-quality free sources in addition to any non-free sources you wish to cite. Nobody has really given a reason why such a suggestion shouldn't be made, other than setting up straw men about how we are telling readers that they should prefer free sources, implying that free sources are always superior to non-free, or are forcing them to find free sources, etc. None of these are true. -- Jrtayloriv (talk) 20:36, 21 August 2010 (UTC)

My point is that the value of the _information_ in a source is independent of how we are able to access it. The value of the information is the most important part of a reliable source. Yes, it helps in the _verification process_ if the information is readily accessible, but that is not enough reason to reduce our standards for the information itself. If we can find a way to not even HINT that we should reduce the quality of the information in deference to accessibility, then I would be okay with it. But doing so will be difficult, I think. --John (User:Jwy/talk) 23:19, 21 August 2010 (UTC)

Straw man. Nobody is suggesting that we reduce our standards or the quality of our information. -- Jrtayloriv (talk) 23:56, 21 August 2010 (UTC)

You are not understanding me. I believe you do not want to reduce the standards for information. I am just worried that any wording that attempts to encourage the use of free sources here (for verification _process_ reasons) will be interpreted as meaning they have an advantage in information quality. The distinction needs to be very clear. I welcome attempts to prove my doubts unfounded. --John (User:Jwy/talk) 03:29, 22 August 2010 (UTC)

I run into many situations in scientific articles where the exact same paper, normally in PDF format, is available on some for-pay site, as well on a free site (typically the author's or their university). I generally switch the URL in the citation in such cases to the free one, to improve source accessibility, which I see as similar to "wayback-izing" a dead link. I would support careful language added to the policy which would recommend that in cases where the same online source document is available on both for-pay as well as free sites, the free URL is recommended. Crum375 (talk) 23:29, 21 August 2010 (UTC)

I think much of the above amounts to the consequence of WP's ongoing use of "verifiable" as a substitute for "verified". If we actually had a mechanism for saying "this paywalled/deadtree source has been verified by user X and user Y to support the statement in the text it is cited against", I would be much more comfortable with the use of such sources. There's a big difference between applying AGF to specific, identified editors' work and applying it to every accumulated edit. The former is a constructive social policy that enables collaboration. The latter flies in the face of our collective experience that vandalism, POV, and other problems routinely affect articles. Absent such a mechanism the use of fair-use quotes from such less-readily accessible works borders on essential to verification. As a purely practical matter, the fact that fewer such sources will be checked does mean the assertions they back are succeptible to more undetected errors. In this context free does mean better.LeadSongDog come howl! 04:19, 22 August 2010 (UTC)

• On the comment by Crum375, we do need to keep in mind that self-archiving is increasingly available even if authors often don't use it - reportedly around 90% (Open_access_(publishing)#Adoption_statistics) of journals allow self-archiving. For example, both Nature (policy) and Science (policy) allow preprints to be posted on the web (with Science seemingly more restrictive), and many other journals allow for postprints. Noting the existence of these full-text freely-available copies and where to find them (Google Scholar being one of the best places to dig them out). II | (t - c) 04:43, 22 August 2010 (UTC)

Some areas have real problems with sources that are trivially available online. A newspaper article about a "medical discovery" is not really an adequate substitute for the peer-reviewed journal article that it's supposedly based upon, even though they are both nominally "reliable" sources under Wikipedia's basic standards.

As a current example, one of today's problems is due to The New York Times apparently issuing a posthumous misdiagnosis that asserts baseball star Lou Gehrig didn't have Lou Gehrig's disease -- on the grounds that some hockey players have the symptoms of a completely separate motor neuron disease. The paywalled source says nothing about either Gehrig or baseball... but you'd never know that, if you focused on the free-beer-online sources.

(IMO this source shouldn't be used at all, since the original paper is definitely WP:PRIMARY literature, but providing a second, "free" source that seriously misrepresents it is far worse than citing the original..) WhatamIdoing (talk) 17:47, 24 August 2010 (UTC)

Nobody was talking about using the New York Times as a free substitute for medical journals. It seems that the proposed guideline has been changed away from saying "equally reliable free source", but if that was re-inserted, it would resolve any concerns with using poor sources like the New York Times. And also, nobody is talking about using sources that misrepresent the primary source anyway -- if the equally reliable free source is not making exactly the same assertion as the non-free source, it shouldn't be used as a citation for that assertion. -- Jrtayloriv (talk) 17:57, 24 August 2010 (UTC)

I don't think you're being very practical about this. This is how it works, out there in the "real" wikiworld: I (temporarily mislay my editorial judgment and) cite the primary paper. You come along and can't get to the source, but you look around and discover the NYT story about the article, and you double-cite it per your proposal. (You are unfortunately unaware that the NYT screwed up in this instance, but it's an honest mistake.)

Next week, a new editor reads just the NYT article, because of the PAYWALL issue. The new editor changes the text of the article to reflect the misinformation in the NYT article, rather than the good information from the PAYWALLed primary source. Now the bad information is "supported" by the peer-reviewed paper -- that the new editor never read, and that the source does not support. The end result is material that is both Not True™ and unverifiable -- and it's got two authoritative-looking citations behind it, and nobody has acted in anything except the best of faith.

IMO we cannot push for double-citations and prevent this from happening -- and this scenario could never have happened, if we didn't push for extra citations to free-beer-online sources. WhatamIdoing (talk) 18:12, 24 August 2010 (UTC)

In response to your scenario, I will repeat my above statement: ...nobody is talking about using sources that misrepresent the primary source anyway -- if the equally reliable free source is not making exactly the same assertion as the non-free source, it shouldn't be used as a citation for that assertion. ... so unless both the PAYWALL source AND the free source were wrong, then we don't have a problem. -- Jrtayloriv (talk) 18:16, 24 August 2010 (UTC)

(P.S. I just realized that the misunderstanding might be my ambiguous use of the term "primary source" above. I should have say "the original PAYWALL source" (I meant primary as in "first"/"original", rather than in the sense of WP:PRIMARY). Was that the problem?)-- Jrtayloriv (talk) 18:18, 24 August 2010 (UTC)

And in my scenario, how exactly do you magically discover that that the NYT article misrepresents the journal article? I have specified that "You are unfortunately unaware that the NYT screwed up." I don't cite the NYT article because it's wrong; you -- not having advanced mind-reading skills, apparently -- cite NYT because you are trying to "helpfully" comply with this proposed rule for double-citations. Consequently, we are talking about (accidentally) citing sources that misrepresent the original paper. WhatamIdoing (talk) 18:48, 24 August 2010 (UTC)

If WP:V is being followed, then the problem you mentioned wouldn't exist. Let me try to explain this more clearly:

For starters, assume that the PAYWALL source is both correct and of very high-quality. Now assume that the assertion that it is backing in the Wikipedia article is an accurate and neutral depiction of what the PAYWALL source actually says. That is, the assertion in the Wikipedia article is not a misrepresentation of the text of the PAYWALL source. Now somebody comes along with a hypothetical free article, of equal reliability. Two things are possible:

1. The free article makes exactly the same claim as the PAYWALL source (i.e. it makes the exact assertion that we are putting in the Wikipedia article), and thus can be used as an additional citation per WP:V. In this case, the free article does not contradict the PAYWALL source, and thus your scenario does not occur.
2. The free article does not make exactly the same assertion as the one in the article and PAYWALL source. In this case, we should not be using it as a source for an assertion it doesn't make, per WP:V (nor does the proposed guideline here suggest that we should). If somebody chooses to include this source anyway, then they are violating WP:V. This is not a problem with suggesting that they find a free alternative to back the assertion. It's a problem with them not finding a free alternative that backs the assertion.

Do you see what I'm saying now? The proposed guideline does not anywhere suggest that "it's a good idea to find a free source, even if it doesn't actually back the assertion being referenced to it". If your scenario occured, that's because of a problem with an editor who decided to ignore WP:V. It's not a problem with the proposed guideline.-- Jrtayloriv (talk) 19:18, 24 August 2010 (UTC)

The "right" way for this to happen: the hapless editor who inserts the "NYT only" information should cite that information ONLY to the NYT (by moving the cite or adding an additional footnote). But the hapless editor has no way of knowing if he is operating under case 1 or 2 above - he is likely, with good faith - to assume he is operating under scenario 1. --John (User:Jwy/talk) 19:26, 24 August 2010 (UTC)

Clarifying my first sentence: If people follow the rules, that's what should happen. The proposed change makes it much more likely that users will (in good faith) break the rules. --John (User:Jwy/talk) 20:25, 24 August 2010 (UTC)

I don't understand what you mean by your first sentence -- could you clarify how that relates to the scenario myself and WhatamIdoing are discussing?

But as far as the editor not knowing whether they are operating under case 1 or 2 above -- I just responded to this possibility above when WhatamIdoing suggested it. Again, this a problem with the editor, not with the proposed guideline. This scenario (accidentally inserting an assertion, referenced by a source that doesn't make this assertion) is already a possibility, under the current guidelines. It is prohibited by WP:V, and should be fixed when an editor notices it. Furthermore, as has also already been stated, this problem is equally likely to occur with a PAYWALL source, and will generally be repaired more slowly than with a free, instantly accessible source. The proposed addition in no way suggests that we start violating WP:V, and in no way increases the potential for this sort of violation to occur. -- Jrtayloriv (talk) 19:51, 24 August 2010 (UTC)

Yes, I agree: The hapless newbie will blindly and blithely assume that all the sources contain equivalent information.

The problem is that sources usually contain more than one piece of information. So I write (in my hypothetical example) "Some forms of MND have been documented in certain professional athletes" (a fact that should be present in both the NYT article and the original paper, so Jrtayloriv might even have verified that the NYT article supported the existing statement).

But the newbie reads solely the NYT article and changes it to the big story, i.e., "A new study indicates that baseball star Lou Gehrig may not have had Lou Gehrig's disease", a "fact" that can only be found in the NYT article. Because Jrtayloriv has (in my hypothetical example) provided this unnecessary double-citation, the newbie is likely to assume that whatever's in the NYT story is actually in the journal article. If we cited only the original (i.e., the sole source actually used in writing the original statement, and thus the sole source appropriate under WP:SAYWHEREYOUGOTIT rules), the newbie would know that s/he didn't know what was in the peer-reviewed paper, and would be more likely to either leave it alone (perhaps muttering to himself about paywalls) or to create a new sentence with a separate citation (if he were already aware of the NYT story). WhatamIdoing (talk) 19:51, 24 August 2010 (UTC)

Your hypothetical double-citation is a violation of WP:V, which the proposed addition does not suggest we start violating. As I said in my response above:

This scenario (accidentally inserting an assertion, referenced by a source that doesn't make this assertion) is already a possibility, under the current guidelines. It is prohibited by WP:V, and should be fixed when an editor notices it. Furthermore, as has also already been stated, this problem is equally likely to occur with a PAYWALL source, and will generally be repaired more slowly than with a free, instantly accessible source. The proposed addition in no way suggests that we start violating WP:V, and in no way increases the potential for this sort of violation to occur. -- Jrtayloriv (talk) 19:55, 24 August 2010 (UTC)

No, you're wrong. Think about it: You happen to know the Truth™ (i.e., that the original scientific paper doesn't mention Lou Gehrig or baseball). The free-beer-online source is wrong -- in part. It could be used to support fact X and misinformation Y, but only fact X is in the original paper.

A less-informed, but good-faith, editor believes the news fiction is completely correct, and adds misinformation Y based on the free-beer-online source. You revert to the accurate version about fact X with a standard edit summary like "Failed verification, Lou Gehrig not mentioned in source" (because it did fail verification, as far as the original, authoritative source is concerned) -- and you get reverted, because some uninformed editor clicked on the free-beer-online source, saw that the NYT article does, indeed, mention Lou Gehrig, couldn't click on the paywalled source but blindly assumed that the NYT got it right (they often do, after all), and decided that you made a mistake.

If you hadn't insisted on citing the news fiction in the first place, then none of this would happen. A handful of people might grump about the paywall, but nobody would have screwed up the content. WhatamIdoing (talk) 20:08, 24 August 2010 (UTC)

I've already responded to this above. This is a problem with the editor making the error -- a problem which is already a possibility under current guidelines, and which the proposed suggestion in no way exacerbates. Your hypothetical error-prone editor could just as easily insert misinformation from a non-free source. Both non-free and free sources can be incorrect, or can be misinterpreted. Including this incorrect information, or including misinterpretations of correct information is already possible under current guidelines, regardless of whether free or non-free sources are used. The cost of the source has nothing to do with reliability or accuracy. If in your scenario, you know that the NYT is incorrect, and have sources to prove it, then you should remove the citation to the NYT article -- nowhere does the current proposal suggest "even if a source is demonstrably incorrect, you should keep it because it's free". Reliability and accuracy are dealt with in WP:V. The proposed guideline in no way overrides or contradicts WP:V, and is dealing instead with the issue of accessibility. The current proposal would only help fix the problem you are mentioning faster, assuming we do what the proposal suggests, which is to include correct representations of high-quality and accurate free sources, by enabling all editors to instantly verify information from such a source. It does not suggest that we do what your scenario is talking about which is citing low-quality incorrect sources, and then having editors make sure that this incorrect information is included in Wikipedia articles. -- Jrtayloriv (talk) 20:29, 24 August 2010 (UTC)

Under the current rules, you aren't spoonfeeding an unnecessary and potentially wrong "free beer online" source to the newbie. We'd cite the pricey-but-excellent source and stop, without implying any equivalence with any other sources. Under the proposed rules, you are setting up the newbie to make this mistake -- and also implying that interested readers aren't capable of asking their favorite web search engines for further information, if they either can't or won't get access to a free-beer source. WhatamIdoing (talk) 05:39, 25 August 2010 (UTC)

As far as free sources being "potentially wrong", I've already said above that the cost has nothing to do with quality. PAYWALL sources can be wrong too. And if it's wrong, we fix it. That's a problem with the editor adding incorrect information, not with the proposed guideline. And it can happen under current rules already. And this proposal in no way increases the probability of that happening. I'll say it again -- Nobody is suggesting including incorrect, inaccurate or low-quality free (or non-free) sources. Nothing in this proposal will increase the likelihood of that happening.. So please stop repeatedly focusing only on the hypothetical case of incorrect free sources -- nobody is suggesting that, and it has nothing to do with this proposal.

As far as being "unnecessary" to provide an additional source -- I disagree. It's very often beneficial to have multiple sources cited for an assertion, so that the reader can be exposed to the deeper and wider context that comes from seeing multiple perspectives on the issue.

As far as including free sources "implying that interested readers aren't capable of asking their favorite web search engines for further information" -- this is not at all true. Does including non-free sources imply that readers can't search for journals on their own? Why do we provide sources at all? Why not just make assertions, and let them Google/JSTOR/etc it if they want to see that it's true? We provide sources so that people can quickly verify the information in the articles, rather than having to do a bunch of research to find out who made the assertion, if it's true, etc. This applies to any sources, free or not. Having free, universally accessible sources just makes this verification process simpler and faster. -- Jrtayloriv (talk) 23:34, 25 August 2010 (UTC)

WhatamIdoing, I share your opinion that peer-reviewed scientific journals should generally be regarded as more reliable than newspapers. But, as Jrtayloriv has explained, the example that you constructed here and spent so many lines discussing is offtopic and does nothing to address the question at hand.

Your scenario would work equally well the other way round: Replace The New York Times by The Times or another paywalled newspaper, and the paywalled academic journal (Journal of Neuropathology & Experimental Neurology, I assume) by an open access one, say PLoS Biology. (Just two months ago, The Times had to issue a major retraction of an article on a scientific topic.) Then you can run the entire argument in the opposite direction: If only that hapless Times reader had been encouraged by WP:V to look for an additional freely acessible source, etc.

Regards, HaeB (talk) 04:19, 26 August 2010 (UTC)

I think this discussion is going in circles. I have not seen significant support for the proposed change and I do not think that new arguments will be offered that would persuade the holdouts. I propose dropping it. RJC ^Talk_Contribs 00:30, 26 August 2010 (UTC)

I don't think you are summarizing the discussion accurately. "not seen significant support" - so you are considering the opinions of NW, SlimVirgin, Jrtayloriv, Crum375 and myself to be insignificant? Care to explain why?

In practice the reliability of Wikipedia is a function of the quality of its sources and the accessibility of these sources, as demonstrated by the hoaxes mentioned above. Fact-checking is a vital part of Wikipedia's process to ensure quality. And as GenericBob said above, not providing an equivalent freely accessible source when it is possible to do so with little effort can often lead to situations where fact-checking is needlessly impeded a great deal.

Regards, HaeB (talk) 04:19, 26 August 2010 (UTC)

Also have to add myself to that list, HaeB. RJC, I don't mean to be mean but you should be careful about misrepresenting 'support' in these types of discussions. II | (t - c) 05:54, 26 August 2010 (UTC)

I apologize if I offended anyone. By lack of significant support I did not mean that the support registered was insignificant, merely that it was not widespread enough to justify changing the page. I wonder, though, whether anyone is on the fence about this such that further discussion will sway them. RJC ^Talk_Contribs 13:41, 26 August 2010 (UTC)

• I am in agreement with RJC and I oppose both the original and "reboot" proposals by NW. A great many WP articles rely on books, old newspapers and other off-line sources, that may not be available online at all or that are only available online for a fee. This applies, in particular, to most newspaper archives. I don't want to have any language in WP:V that, directly or indirectly, discourages or devalues the use of such sources. The original idea of NW of encouraging the use of open access scholarly journals is also, at this point in time, problematic. I am an academic myself and, in my observations, it is far too early to start encouraging references to open access journals in preference over traditional printed ones. In my own field, mathematics, almost all open access journals at this point are very low quality journals - basically paper mills with extremely perfunctory/pro forma peer review that make a profit by charging rather exorbitant per page publication fees to the authors. I suspect that the situation is similar in many of the other fields. It may well be that in a few years things will change, but not yet. Nsk92 (talk) 06:23, 26 August 2010 (UTC)

• There is nothing in the proposal that discourages the use of any form of non-free source. There is also nothing in the proposal suggesting people use a low-quality non-free source. If a high-quality free source does not exist, then a free source should not be used.-- Jrtayloriv (talk) 07:21, 26 August 2010 (UTC)

• That is a matter of interpretation. I am pretty sure that any such language, as that used in the proposal, will in practice be interpreted by many as giving some sort of preference and weight to free online sources. As a practical matter, the proposal is also redundant. In practice most editors use various types of google searching first, when looking for sources, and if they find free online sources, they use them. I do not feel it is at all necessary to give any extra encouragement for the use of such sources. On the other hand, any form of discouraging the use of off-line sources is, in my opinion, highly problematic and likely to cause more problems than it solves. Nsk92 (talk) 07:48, 26 August 2010 (UTC)

• Also, the proposal seems to be based on a fairly naive view of how sources are used. A source is not always simply used to verify some basic fact (such that a particular person was born on such and such date). Often sources are used to represent different views of various experts on a particular topic, involving a polemic on that topic; in such a situation it is a bad idea to give extra ammunition to anyone who will want to argue that free online sources in such a polemic should be given extra weight. It is also often the case than an authoritative text on some topic is an off-line book or article - in such a case the use of such an off-line source should be given priority over the use of free on-line sources. And so on. I can foresee many types of situations where an explicit encouragement of the use of free online sources over other types of sources may lead to problems. On the other hand, as I said above, IMO in practice any such encouragement is unnecessary since people tend to try to find free offline sources first anyway (in fact perhaps more than they should - there are many cases where it is really preferable to expand some time and go to the library to look-up some comprehensive authoritative off-line source on a particular topic). Nsk92 (talk) 08:07, 26 August 2010 (UTC)

By that logic, WP:NONENG would need to be immediately abandoned because of systemic bias concerns. And the proposal does not apply at all to "the case than an authoritative text on some topic is an off-line book or article". Regards, HaeB (talk) 18:56, 26 August 2010 (UTC)

Nsk92, you are making a lot of unsubstantiated and far-reaching claims here. Which of the journals listed here are "basically paper mills with extremely perfunctory/pro forma peer review that make a profit by charging rather exorbitant per page publication fees to the authors"? Do you regard Geometry & Topology and the Bulletin of the American Mathematical Society as "very low quality journals"? Regards, HaeB (talk) 13:09, 26 August 2010 (UTC)

Geometry and Topology is an excellent journal (I have published there two papers myself -:), but it is not an open access journal in the traditional sense, see subscription information here[1]. The journal does charge for print and electronic subscription, although electronic access becomes free 3 years after publication. I am not sure what the subscription model with the Bulletin of AMS is but I do receive it every month, together with AMS Notices, as an AMS member. I think it is largely financed by the AMS membership fees. It probably is open access but among the math journals this is rather an exception. I know that LMS also has one free journal LMS Journal of Computation and Mathematics, where both access and publication are free. Note that both of these examples (Bulletin of AMS and are fairly LMS Journal of Computation and Mathematics) are exceptional in the sense that they don't charge for either publication or access and are completely free to both authors and readers. Typical open access journals charge significant publication fees to the authors. I don't know of any good math journals of this latter kind and every single one that I have seen does look like a paper mill with exorbitant publication fees. Nsk92 (talk) 14:41, 26 August 2010 (UTC)

The Directory of Open Access Journals lists 160 math and stats journals. I'm pretty sure we can find amongst those some which are free to both readers and authors. LeadSongDog come howl! 17:16, 26 August 2010 (UTC)

I named Geometry & Topology as an example because it is contained in the linked list of free journals, provided by "The Mathematical Survey" project on the UC Berkeley website. The precise publication process does not matter, the point is that the vast majority of the journal's issues are freely available. In fact, the example illustrates another fallacy in the "paywalled=better" assumption: Do the G&T articles suddenly become less reliable as soon as they are made freely available after 2-3 years?

"...exceptional in the sense that they don't charge for either publication or access and are completely free to both authors and readers. Typical open access journals charge significant publication fees to the authors" - is this statement based on personal impressions, too, or on actual statistics? this study found that "most Full Open Access journals (52%) do not in fact charge any sort of author-side fees".

The Notices of the AMS are freely available on the web, too (for a full PDF of the August 2010 issue, or separate articles, go here).

Regards, HaeB (talk) 18:56, 26 August 2010 (UTC)

I was in fact aware that Notices of AMS is available online for free (I used it as a ref in some bio articles about mathematicians). But Notices of AMS is not really a journal in the traditional sense. Most of what it publishes is information (about AMS meetings, other math conferences, AMS elections, various prizes etc). They do publish some math articles but only survey/expository ones, no new research. Bulletin of AMS also publishes mainly book reviews and survey articles, not new research. The main AMS research journals, such as the Journal of AMS and the Transactions of AMS are not free. In general, fairly few outlets can afford to publish a totally free journal (both to authors and readers). Some big math societies (AMS, LMS) can do it, on occasion, because they can rely on their membership fees and income from their other non-free publications, but even for AMS and LMS almost all of their journals are not free and the free ones are sort of stand-alone experiments. Some math institutes might be able to publish a completely free journal and maybe an occasional math department may do the same where the university administration is particularly enlightened and is willing to cover the cost. It does cost some non-negligible amount of money to publish a math journals and the totally free ones are quite rare. Nsk92 (talk) 07:18, 28 August 2010 (UTC)

Regardless of the existence of specific counterexample to Nsk92's assertion, it is by and large the case that open access material is of a substantially lower quality than content behind a paywall. But, even excepting this particular point, Nsk92 seems to have struck the proverbial nail squarely on its head. We scarcely need to do more to encourage the use of online open access content. What we should be doing is encouraging the use of the best quality sources, regardless of the mode of access—free, paywall, or dead-tree should not even enter into the calculation. Sławomir Biały (talk) 13:18, 26 August 2010 (UTC)

What we should be doing is encouraging the use of the best quality sources, regardless of the mode of access. Give that man a cigar. (Or a beer, or a stuffed animal, or whatever he likes.) Short Brigade Harvester Boris (talk) 14:49, 26 August 2010 (UTC)

Unfortunately "anyone can edit" says nothing about living around the corner from the National Archives. Accessibility does matter, and the best sources are occasionally obscure, whether by modern paywalls or simply by pre-Gutenberg rarity. We need to have a means to ensure when obscure sources are used that they are well used both in good faith and in balanced representation. This means we need a way to have multiple editors vet such content. This is not at all an argument against using such sources, but a caution on how we ensure that WP:V is more than hypothetical. LeadSongDog come howl! 17:16, 26 August 2010 (UTC)

The "scarcely need to do more" assumption is debunked quite nicely by the comments of the very user you are applauding: If even an experienced mathematician such as Nsk92 is not aware that (or not sure whether) two of the most well-known journals in his field - Notices of the AMS and Bulletin of the AMS - are freely available online, even though he is subscribing to their print versions, it certainly cannot be assumed that freely available versions of paywalled/print sources will always get noticed automatically.

Regards, HaeB (talk) 18:56, 26 August 2010 (UTC)

I think you must have missed my point altogether, which had nothing to do with whether some journals were open access or not. My point was that Wikipedia is already drowning in rubbish sources pulled from the net. We really don't need to encourage more of this. Rather we should look for the best sources, regardless of what kind of access is associated with them. I know the proposal is supposed to be that only somehow "equivalent" sources get used. But that's really the wrong way of looking at things. Choose the best sources, period. If there are sources of comparable quality, and one of them is freely available, then I don't think any editor would object to adding an additional reference for verification, unless of course there was some question about the quality of the source. That makes the proposed addition rather redundant with editorial judgment, and more ammo to the guy who's gaming to add questionable sources. And it is a sad fact that many, perhaps most, Wikipedia editors seem to lack the judgment necessary to make this determination. For instance, I've encountered more than one regular editor who honestly believes that MathWorld is an imminently trustworthy source for mathematics. In short, I see no reason that we should have, enshrined in policy, a sanction to add still more crap from the net. Sławomir Biały (talk) 21:29, 26 August 2010 (UTC)

I sympathize with your grievance about some editors' attitude towards Planetmath (I remember a discussion which matches your description), but I really can't see how it relates to the present debate - no one proposed to add a statement that says open access sources should be regarded as more reliable than they are regarded now. And please be aware that we are talking about a general policy here, and also consider cases from entirely different areas, such as "Handel" above, which convincingly demonstrates that citations naming reputable offline sources can be very dangerous, too.

"If there are sources of comparable quality, and one of them is freely available, then I don't think any editor would object to adding an additional reference for verification" - people do in fact often object if there are several references for the same statement, and reduce them to one (e.g. citing the WP:CLUTTER essay). But in addition to that, it is good to make editors aware that citing equally reliable sources which are freely accessible can improve the quality of the encyclopedia, by making fact-checking easier for others. I am sorry if I had misunderstood you, but your statement "We scarcely need to do more to encourage the use of online open access content" seemed to assume that in a case where there is an identical open access version available (such as for the Bulletin and the Notices of the AMS, or with green OA), editors will automatically be aware of it and cite it. As demonstrated above, this is not true.

Regards, HaeB (talk) 06:03, 28 August 2010 (UTC)

On Thursday, during the above debate, I asked at the WikiProject Mathematics whether Nsk92's above statement on open access journals (that formed a central argument in his objections to the proposal) reflected a widely held view. Eight users responded, none of them supporting Nsk92's view, and some disagreeing quite strongly (e.g. Boris Tsirelson, who said he was "proud to publish good articles in The New York Journal of Mathematics and Probability Surveys"). The general consensus seems that journals should not be judged on whether they are freely accessible or not, but according to their editorial policy (peer-reviewed or not), reputation of their editors, etc.

Regards, HaeB (talk) 06:15, 28 August 2010 (UTC)

I have just now seen that WikiProject_Mathematics thread and have commented there. The examples that you mention The New York Journal of Mathematics and Probability Surveys are totally free journals that do not charge anything to either authors or readers. For math journals that is still pretty rare. Most open access math journals charge substantial author publication fees. I receive e-mails with announcements of such journals being launched a few times per month and have even been asked to join the editorial boards of a few of them (which I declined). I don't know of any good math journals of this kind while they keep popping up like mushrooms. Nsk92 (talk) 07:23, 28 August 2010 (UTC)

re-restatement

To answer some of the concerns that other editors have expressed above, I wanted to suggest another restatement of the proposal, that seems to deal with them:

However, if an equally reliable and freely accessible source exists which makes exactly the same assertion as the non-free source, then it would be helpful (but is by no means required) to cite the free source in addition to the non-free source. This can assist with verification for editors who cannot gain access to the non-free source. It does not imply that freely accessible sources are generally superior or preferable to non-free sources.

This makes it clear that:

1. The source must be equally reliable -- it cannot be a free source of lower quality.
2. The source must make exactly the same assertion (this is redundant, per WP:V, but it seemed to be a frequent concern, so we've made it explicit here).
3. The source is provided in addition to the non-free source, not as a replacement for it.
4. It is not required to add such a source if you don't feel like it.
5. It makes it clear that the purpose is to assist with verification.
6. It makes it clear that we are not implying that free sources are superior or preferable.

Anyhow, I think this could be tweaked, and I think a lot of it is actually redundant, but it seems to me to clear up the concerns that have been voiced above. Opinions? -- Jrtayloriv (talk) 20:37, 26 August 2010 (UTC)

I'd be fine with this, though I'd tighten it. No need for "this can assist" etc. SlimVirgin ^{talk|contribs} 21:15, 26 August 2010 (UTC)

I still would prefer the language from NW's original proposal ("equally reliable and more easily accessible") without the added restriction "makes exactly the same assertion".

However, considering the many objections to NW's proposal and the lack of opposition to yours, I just added your sentences, with the following small modifications that do not touch the six points above:

• "freely accessible online" instead of " freely accessible" (i.e. not just freely accessible in one library on another continent)
• "This can improve the efficiency of the verification process" instead of "This can assist with verification for editors who cannot gain access to the non-free source": That wording could have been misunderstood - after all, verifiability implies that in principle everyone can gain access to the source. And even an editor who has free access to a paywalled journal at his workplace (university or company) might hit the paywall when checking Wikipedia references in his leisure time at home, so he too can benefit from the additional citation.

I think it would also be worthwhile to explicitly mention self-archiving (green OA), i.e. the fact that many journals which are not freely accessible nevertheless allow authors to put copies of their articles online, but I haven't added language to that effect yet.

Regards, HaeB (talk) 05:52, 28 August 2010 (UTC)

I have just reverted the edit of HaeB inserting the above text in WP:V. I still find the change objectionable and unnecessary. There is no need for any further encouragement of the use of free sources. Moreover, the proposal's language assumes a rather naive view of how sources are used. Nsk92 (talk) 06:47, 28 August 2010 (UTC)

• There is no need for further encouragement of the use of free sources -- Sure there is. It makes it easier to verify information, thus leading to a more accurate encyclopedia. It is a favor to our readers (the people we are writing the encyclopedia for) -- the vast majority of whom do not have easy access to journal articles, university libraries, etc., but who (obviously) have a connection to the internet. If we provide them high quality sources that they can instantly access, they can (a) rest assured that the encyclopedia is accurate, (b) fix inaccuracies that they would otherwise not discover. If we have only offline/paid-access sources for an assertion, then we are leaving the burden of fixing problems on the small number of editors who do have access to these sources AND take the time to actually get the sources. If we also add high-quality free sources, the burden is spread over an immensely larger number of editors (i.e. every editor).
• the proposal's language assumes a rather naive view of how sources are used. -- Could you please elaborate on this? What is "naive" about it? -- Jrtayloriv (talk) 07:13, 28 August 2010 (UTC)

• Regarding the first point. What I mean is that in practice WP editors tend to try to find free online sources first anyway - they do various kind of google-searching first when looking for sources. Regarding the second (and to some extent the first) point. Our goal should be to encourage the use of the best sources, not the most quickly accessible sources. It is the use of best sources that leads to a more accurate and high quality encyclopedia. In my experience, the best sources, such as authoritative books or articles on a given topic, are often off-line sources where one needs to go to the library to read them. Also, sources are not simply used to verify some simple basic fact (that someone was born on such and such a date or that some event took place on some particular date). Sources are also, and maybe even more often, used to discuss various points of view in a particular debate. What does a particular scientific experiment mean? What does the archeological evidence indicate about dating a particular event? Etc, etc. In such cases the most relevant sources may not be available online for free or available online at all, and it is not at all a simple matter of giving a "free online" reference next to an off-line/non-free reference. Nsk92 (talk) 07:37, 28 August 2010 (UTC)

I agree with Nsk92. Also, he and I have both made the point that online sources already receive undue priority among Wikipedia editors. The "systemic bias" issue that Nsk92 alludes to is also a very real concern. However, the essence of the issue remains: (1) The encyclopedia is helped most by the best sources, not the most accessible. Method of access should not be a concern at all. Also, as I pointed out above, the language of ensuring that the free and non-free sources be somehow "equivalent" will have no practical effect. Most Wikipedians editing outside their topic areas will be totally unable to determine whether two sources are of equal quality. But, in any event: (2) Typically the best sources in many topic areas remain dead-tree sources and sources behind a paywall. In some areas (mathematics, for instance), we really need to get more of these high quality sources as references. Adding language to policy that appears to favor free online sources over other sources is counterproductive. Sławomir Biały (talk) 12:07, 28 August 2010 (UTC)

Since HaeB referred to the fact that we had not objected to this specific proposal, I should register that I too agree with Nsk92 and oppose the change. RJC ^Talk_Contribs 14:26, 28 August 2010 (UTC)

One needn't insist on an either/or. The better approach is to request an accessible source be added where possible. It is an unfortunately common occurrence to cite a hard-to-obtain source as support for arguments that are not those of the source. Or, the source supports a minor point, but not the thesis. So verifiability is impacted by unavailability, and I'd suggest that when a hard-to-obtain source is used, the supporting material should be placed in a verbatim quote as some insurance that the writer actually has read the source correctly. Brews ohare (talk) 14:26, 16 September 2010 (UTC)

Academic and media sources

We currently say, Academic and peer-reviewed publications are usually the most reliable sources where available, such as in history, medicine, and science, but they are not the only reliable sources in such areas. Material from reliable non-academic sources may also be used, particularly if it appears in respected mainstream publications. Other reliable sources include university-level textbooks, books published by respected publishing houses, magazines, journals, and mainstream newspapers.

This has been a long-standing consensus version, and I approach what I am going to say with some trepidation. The thing is, academic and peer-reviewed publications are generally held to be more reliable than media sources. This view is reflected in WP:MEDRS, which explicitly disqualifies media sources as preferred sources, but it surely applies to other topic areas (art, literature, religion, politics, media studies, business, information technology, industry, engineering etc.) as much as it does to medicine. (For one, I can think of a few press reports on Wikipedia that contained fundamental errors – some of them documented in Signpost articles – that an academic researcher would have been unlikely to make.)

The present wording, "Academic and peer-reviewed publications are usually the most reliable sources where available" appears to say as much. But it first restricts it to topic areas like "history, medicine, and science" and then seems to level the playing field altogether: "but they are not the only reliable sources in such areas" -- making editors think there are no differences in reliability between scholarly works and press or online articles, and no reason to make an effort to seek out the former in preference over the latter. This is reflected in the sourcing of many of our articles.

I would like this policy to direct editors to make a bit more of an effort to consult books and scientific journals in areas where they are available. Media and online articles are valuable, but they cannot replace books and scholarly works; and if we ignore the scholarly literature in topic areas where it is available, we are not representing the sum of all human knowledge. Thoughts? --JN466 15:00, 12 September 2010 (UTC)

The fact that one online source (academic or other) makes a mistake should not, in my opinion, lead us to conclude that all online sources are inherently unreliable. So I question your two mentions of the word "online". But I agree that the wording in question is inadequate. The idea that one category of source is more or less reliable than another (as opposed to a statement about what is and is not a reliable source) is raised, but without producing a conclusion. By the time we reach the end of the long list of qualifiers, usually ... the most reliable ... where available ... but they are not the only ... may also be used ... particularly if, there's something there for everyone, and it seems just about anything goes. This is, after all, a policy document: the passage should make plain statements. Firstly, what is and is not a reliable source, and secondly, what preferences apply should sources in more than one category exist. PL290 (talk) 10:05, 13 September 2010 (UTC)

Thanks, you summarised my concern well. I agree that I did not express myself clearly when I referred to online sources -- I meant material that is only published online, by websites that have enough basic editorial oversight to qualify as low-end RS, but are not the work of major media or academic institutions. --JN466 12:47, 13 September 2010 (UTC)

Proposal

Here is a proposal:

Academic publications, such as peer-reviewed journals and books published by well-regarded academic presses, are usually the most reliable and most valuable sources in the topic areas where they are available. However, they are not the only reliable sources in such areas. Non-academic sources may also be used, particularly if they are respected mainstream publications. This includes books published by reputable publishing houses as well as mainstream newspapers, magazines, and journals. Electronic media may also be used, subject to the same criteria.

This would replace the following policy paragraph:

Academic and peer-reviewed publications are usually the most reliable sources where available, such as in history, medicine, and science, but they are not the only reliable sources in such areas. Material from reliable non-academic sources may also be used, particularly if it appears in respected mainstream publications. Other reliable sources include university-level textbooks, books published by respected publishing houses, magazines, journals, and mainstream newspapers. Electronic media may also be used, subject to the same criteria.

Also see Wikipedia:IRS#Some_types_of_sources. I do not think the proposed policy wording would require any change in the WP:IRS guideline. Views? --JN466 13:16, 13 September 2010 (UTC)

Agree in principle with the proposal. One aspect that could usefully be tightened up is making explicit in this paragraph the requirement in the first paragraph that we consider "the creator of the work (for example, the writer)". Suggest "This includes books by reputable authors published by recognised publishing houses...". Even the most reputable publishing houses sometimes publish questionable fringe views, if only as a talking-point. For example, The Design Inference. . . dave souza, talk 13:31, 13 September 2010 (UTC)

I would oppose any wording that downgrades non-academic sources. Things are bad enough at the moment, with some editors trying to impose scientific point of view. I wouldn't want to see the policy encourage it, because it risks giving the green light to leaving out POVs that for various reasons might not be found in scholarly texts, but which are nevertheless regarded as important by reliable sources. SlimVirgin ^{talk|contribs} 13:40, 13 September 2010 (UTC)

OK, Slim (per medical articles), but consider the counterexample occurring at Hugo Chavez. The academic press so far is lagging, and reflects largely left-leaning publications (in fact, far left), yet the editors who own the article reject all other mainstream pubications-- even though dozens, scores and hundreds contribute to due weight of mainstream reliable views-- with the claim that only academic press should be used. (And curiously, some of the same editors who frequently decry "US or "corporate" bias in the mainstream news media are more than happy to cite text to the partisan website, Venezuelanalysis.com.) SandyGeorgia (Talk) 14:03, 13 September 2010 (UTC)

Hmmmm ... on re-reviewing the two sets of wording side-by-side, I do see that JN's proposal is downgrading the wording, and agree with Slim. Medicine and science have some differences in this sense from other areas, wrt RECENTISM, sample size, replication of primary studies, etc. SandyGeorgia (Talk) 14:10, 13 September 2010 (UTC)

Generally speaking, underrepresentation of academic sources is a more widespread systemic problem in Wikipedia than overreliance on such sources, but it depends on the topic area. Broadly speaking, bearing in mind the entire range of scholarly research, there is widespread consensus in society that scholarship – generally speaking – produces better informed, more detailed and more reliable work than journalism, and the policy should reflect this. Academic writers are subject matter experts who have invested years of training in the fields they cover; the same cannot be said of journalists covering the same field. Having said that, I would strongly oppose any editor arguing that only academic sources should be used, and that the entire public discourse that happens in the media, and the findings of the media's investigative research, should have no place in Wikipedia – especially in a field like politics. I don't think the proposed wording would support any editor in making that argument. It states clearly that non-academic sources may also be used. --JN466 14:23, 13 September 2010 (UTC)

As always, we need to find the right balance. I have seen abuses in both directions. There are attempts to give a lot of weight to scientific claims that contradict the findings of actual scientists, merely because their supporters are making a big splash in the general media. And there are attempts to exclude mention of highly notable disputes because they are related to science and there are no scientific publications on them. Neither is OK. When Anne Elk's theory of brontosauruses becomes the latest craze and all major newspapers write that this "theory" revolutionises palaeontology, then the following obviously will apply:

• Any factual, scientific claims about actual dinosaurs must be sourced with a bias for academic sources.
• Academic sources are fine though unlikely to exist for the dispute between Anne Elk and the palaeontological establishment. So we use general media for that, taking care not to automatically accept what they say about the science. Hans Adler 14:11, 13 September 2010 (UTC)

My point is that we shouldn't report what the laypress says about primary studies-- that haven't been subjected to secondary review or replication subject to larger sample sizes-- in the fields of medicine and science, except under very special circumstances, while we have literally hundreds of mainstream reliable sources presenting human rights abuses, consolidation of power, rampant crime and corruption, and decline in democracy in a current politician, which hasn't yet made it into the academic press. We must accord due weight to mainstream, published, reliable points of view. And, we must also weigh biases in academic presses. SandyGeorgia (Talk) 14:18, 13 September 2010 (UTC)

That is a sensible point, Hans. I would argue that the dispute between Anne Elk and the palaeontological establishment is a topic area that has not (yet) been covered by scholarly sources, and that this is taken care of by "in the topic areas where they are available". I would not mind adding a subclause that makes that clearer. --JN466 14:23, 13 September 2010 (UTC)

And your proposal allows the Chavez article to continue its bias against hundreds of mainstream reliable sources over what has been published so far by the left-leaning academic press, hence I oppose. Specific guidelines for medicine and science articles address sources in those types of articles, and contemporary politicians are more likely to be covered by mainstream reliable sources such as the news media. SandyGeorgia (Talk) 14:27, 13 September 2010 (UTC)

We cannot let the problems of the Chavez article drive the wording of a global policy. On the other hand, I don't mind adding a sentence to the effect that journalistic sources are indispensable to cover current affairs and notable controversies in any field, be it science or politics. Might that be a way to address your and Slim's concerns? --JN466 14:31, 13 September 2010 (UTC)

I edit mostly medical articles, and only one politician, while Slim edits many more controversial contemporary topics; I'd like to see what she has to say about how representative the Chavez situation is of other similar contemporary articles. I suspect it's pretty common, and the "academic press" argument here is being used to suppress mainstream points of view. I suspect we have similar at the Catholic Church, where recent sexual abuse issues haven't yet been covered by scholarly sources. SandyGeorgia (Talk) 14:34, 13 September 2010 (UTC)

(ec) The thing we have to be wary of is writing policy with only some examples in mind, because this would be a sweeping change across many different kinds of articles (and therefore many different kinds of POVs), and no matter how carefully you tweak the writing, you risk it being used to exclude legitimate POVs. Two examples I can think of:

(a) Editors who were followers of a cult leader tried to argue that only academic sources should be used about that figure, and that the Los Angeles Times was therefore not a reliable source. The reason, as I recall, was that the LA Times had published a photograph of the leader's enormous house.

(b) The Jesus articles: editors regularly argue there that only academic sources should be used. This sounds reasonable until you realize that the specialist academic sources are mostly biblical scholars, and the biblical scholars are mostly religious people, including high-ranking church figures.

The danger, as Sandy points out, is that reliable and important POVs are excluded when you focus on scholarly sources, because the academic press is slow to publish, or because it's the academic views themselves that are being criticized. We have to make sure this policy safeguards all reliable points of view, not just academic ones. SlimVirgin ^{talk|contribs} 14:36, 13 September 2010 (UTC)

Jayen, can you say exactly what benefit you would see in changing the wording? SlimVirgin ^{talk|contribs} 14:38, 13 September 2010 (UTC)

I would like to see editors make it a matter of course to research the available scholarly literature on a topic they are writing about in Wikipedia, to look for sources in google books and google scholar, as well as newspapers and websites. We are losing much valuable content when editors don't do this, and it takes a little more effort. --JN466 14:53, 13 September 2010 (UTC)

The presence of intellectual-methodological-theoretical-ideological nexuses in the social sciences and humanities is relatively non-controversial institutionally. Universities tend to be loathe to fire controversial opinion leaders of the left or right, as long as they make their required publication outputs. I find controversy over recent events to be slightly humorous (give it fifty years, we'll find out what really happened). It sounds like there is a discipline specific problem in non-historical social sciences which excite controversy in the general public. Losing the emphasis on HQRS for fields like labour history, industrial relations, human resources, economic history, history and philosophy of science, psychology would be catastrophic, and invite the newspapers into a domain where they should not be present. But at the same time we have a problem with "contested" fields of politics. Drop the barrier, and watch the non-academic "think-tanks" and stable reputable (but yet non-RS for WP purposes) partisan sources line up for a bite at the cake. A problem. Fifelfoo (talk) 14:38, 13 September 2010 (UTC)

The think tanks already have a large bite of the cake, even when they are clearly partisan and to the exclusion of other reputable sources published in sources like Foreign Policy and Foreign Affairs; hence, we need to give more beef to other mainstream reliable sources, not less. SandyGeorgia (Talk) 14:55, 13 September 2010 (UTC)

Scholarly publications are certainly important sources, but they aren't perfect by any means. To repeat some points already made here, they can be years or even decades behind other sources, they tend to be focused on disproportionately on academic issues, and they have biases of their own. Even within a single article, scholarly papers may be the best available sources for a point of theory while newspapers may be the best sources for events or quotations. I also notice that the proposed wording drops the qualifier, "published by well-regarded academic presses". There are peer-reviewed journals with poor reputations, so we should avoid implying that they are all of the same high quality.   Will Beback  talk  21:19, 16 September 2010 (UTC)

Will, you have that the wrong way round. The proposed wording introduced the qualifier "published by well-regarded academic presses" (red is the current policy wording). Otherwise I agree with your point; scholarly and press sources are often complementary. I would also be interested in your views on the discussion of Proposal 4, below. --JN466 22:06, 17 September 2010 (UTC)

Proposal 2

Academic publications, such as peer-reviewed journals and books published by well-regarded academic presses, are usually the most reliable and most valuable sources in the topic areas where they are available. However, they are not the only reliable sources in such areas. Non-academic sources may also be used, particularly if they are respected mainstream publications. This includes books published by reputable publishing houses as well as mainstream newspapers, magazines, and journals; mainstream media sources in particular are indispensable to adequately reflect public opinion and current affairs in Wikipedia. Electronic media may also be used, subject to the same criteria.

Does that address your concerns? --JN466 14:49, 13 September 2010 (UTC)

Can you please use Wikimarkup to highlight the changes? SandyGeorgia (Talk) 14:58, 13 September 2010 (UTC)

I've now highlighted the added passage in bold font. --JN466 15:02, 13 September 2010 (UTC)

But wasn't something dropped? If so, could you show it as struck? SandyGeorgia (Talk) 15:06, 13 September 2010 (UTC)

The bolding only highlighted the change compared to Proposal 1 above. Compared to the present policy wording, these are the changes:

Academic and peer-reviewed publications, such as peer-reviewed journals and books published by well-regarded academic presses, are usually the most reliable and most valuable sources in the topic areas where they are available, such as in history, medicine, and science, but. However, they are not the only reliable sources in such areas. Material from reliable Non-academic sources may also be used, particularly if it appears inthey are respected mainstream publications. Other reliable sources include university-level textbooks, books published by respected publishing houses, magazines, journals, and mainstream newspapers.This includes books published by reputable publishing houses as well as mainstream newspapers, magazines, and journals; mainstream media sources in particular are indispensable to adequately reflect public opinion and current affairs in Wikipedia. Electronic media may also be used, subject to the same criteria. --JN466 15:21, 13 September 2010 (UTC)

Policy

Jayen's proposal

Academic and peer-reviewed publications are usually the most reliable sources where available, such as in history, medicine, and science, but they are not the only reliable sources in such areas. Material from reliable non-academic sources may also be used, particularly if it appears in respected mainstream publications. Other reliable sources include university-level textbooks, books published by respected publishing houses, magazines, journals, and mainstream newspapers. Electronic media may also be used, subject to the same criteria.

Academic publications, such as peer-reviewed journals and books published by well-regarded academic presses, are usually the most reliable and most valuable sources in the topic areas where they are available. However, they are not the only reliable sources in such areas. Non-academic sources may also be used, particularly if they are respected mainstream publications. This includes books published by reputable publishing houses as well as mainstream newspapers, magazines, and journals; mainstream media sources in particular are indispensable to adequately reflect public opinion and current affairs in Wikipedia. Electronic media may also be used, subject to the same criteria.

J, could you say which parts would imply what kind of change? SlimVirgin ^{talk|contribs} 15:47, 13 September 2010 (UTC)

• Some changes are just streamlining the wording; e.g. "university level textbooks" are subsumed under academic sources and don't need explicit mention. The listing of "other reliable sources" flows better and is more compact.
• Adding "most valuable" and removing the limitation "such as in history, medicine, and science" makes clear that if I write about Doris Lessing, Deep Purple, Monty Python, Midnight in the Garden of Good and Evil, Harry Potter, New Age or the European Exchange Rate Mechanism, it is just as good an idea to look for books and scholarly sources as it is when writing about physics, or medicine.
• The explicit mention of mainstream media as an indispensable source for public opinion and current affairs addresses the fact that academic publishing generally lags five or ten years behind the times, and provides a stronger basis for covering notable controversies. At the same time, it weakens the case of those who would like to use the press as a source for science proper, by indicating what the main strength of press sources is. --JN466 16:08, 13 September 2010 (UTC)

Some thoughts:

• A university-level textbook needn't be an academic source.
• I would say adding "most valuable" risks over-valuing them compared to other sources.
• I have no objection to adding the mainstream media sentence, but I would want to see it worded a bit differently, more along the lines of your earlier suggestion, e.g. "mainstream media sources in particular are indispensable for the coverage of controversies, whether in science, politics, or elsewhere." But I would worry about anything that implies they're not good for other things, because we can't foresee all circumstances.
• I wouldn't object to removing "such as in history" etc.

As for your concern about encouraging editors to look for better sources, perhaps we could just add some words to the existing policy to remind them to do that? SlimVirgin ^{talk|contribs} 16:22, 13 September 2010 (UTC)

• I don't mind dropping "most valuable"; "most reliable" can stand on its own.
• A sentence on looking for book and scholarly sources would be very useful.
• Do you think we ought to explicitly mention university-level textbooks? I thought between "academic sources" and "books published by respected publishing houses" we have it covered.
• For the media sentence, I offer this wording: ... respected mainstream media sources in particular are indispensable to adequately cover public opinion, current affairs and controversies, whether in science, politics, or elsewhere. --JN466 19:31, 13 September 2010 (UTC)

I am opposed to dropping the reference to medical and science articles; they have their own carefully developed guideline pages, which reflect and complement WP:V, and dropping them will encourage overreliance on primary sources. And I would strongly oppose any wording like "mainstream media sources in particular are indispensable for the coverage of controversies, whether in science, politics, or elsewhere", considering some of the laypress misreporting of medical controversies. SandyGeorgia (Talk) 02:39, 14 September 2010 (UTC)

The three examples that policy currently mentions are history, medicine, and science. There is nothing special about these fields compared to other fields that are the subject of academic study. As for these areas having their own guideline pages, that is not true:

• There is no separate RS guideline for history articles; attempts to start one at Wikipedia:Reliable_sources_(history-related_articles) never got off the ground.
• Wikipedia:Identifying_reliable_sources_(natural_sciences) was a proposed guideline that has so far failed to win community support.
• WP:MEDRS is an actual guideline that will stand regardless of what we do here. Commenting on press sources, it says, "the high-quality popular press can be a good source for social, biographical, current-affairs and historical information in a medical article. For example, popular science magazines such as New Scientist and Scientific American are not peer reviewed but sometimes feature articles that explain medical subjects in plain English. As the quality of press coverage of medicine ranges from excellent to irresponsible, use common sense, and see how well the source fits the verifiability policy, and the general reliable sources guideline." That is compatible with the wording proposed here.

More generally speaking, we shouldn't write a policy that allows editors to define areas where the press should be ignored as uninformed, versus areas where the press must be used as a vital counterbalance to systemic bias in scholarly sources, according to their personal preferences, while all the while claiming that policy backs them up. Making the wording here so elastic that anyone can use it to justify anything does not help the project; it just creates endless strife. We have to strive for some measure of consistency, at least in a global policy. So, use science sources for science proper, and use the press to cover notable controversies, social aspects and current affairs. Even if the quality press is wrong, according to an editor's view (cf. the endless debates about global warming skepticism), if it is a notable controversy, it should still be covered -- as should the criticism of press coverage from sources that claim to be better informed.

And this does not mean that if a tabloid announces yet another new miracle cure for cancer, that it must be covered in Wikipedia. At the same time, it does mean that if Hugo Chávez has an exceedingly poor reputation among large sections of the high-quality mainstream press, then it should be covered in his biography, even if the published academics all love the man. --JN466 03:24, 14 September 2010 (UTC)

I did not comment on history (not my area); here is Science. While I appreciate that some of these changes may address the ongoing POV and disruption at Chavez, I doubt that they will have any effect, as editors there routinely cleanse any text unsupportive of Chavez, even if from scholarly sources, and I oppose any wording that reduces the importance of MEDRS, as a carefully developed guideline that complements this policy page. SandyGeorgia (Talk) 10:47, 14 September 2010 (UTC)

I wonder what has changed since July, when we last discussed this section at great length and came to the current wording. This is especially as what people were trying to accomplish then (as now) is really more a matter of WP:NPOV and its associated guidelines (WP:UNDUE, WP:FRINGE, etc.) than of verifiability. Why hasn't this been settled already? Or are we simply hoping that a different cast of editors will reach a different conclusion? RJC ^Talk_Contribs 04:18, 14 September 2010 (UTC)

Do you mean this discussion in July? The present wording essentially dates back at least to April. --JN466 05:07, 14 September 2010 (UTC)

The fact that it's been discussed at great length in the past confirms to me what I said in my response to the original post in this master thread: it's trying to achieve too much, and ends up full of qualifiers and ultimately lacking a clear policy statement. Perhaps it's trying to act as both policy and guideline. In my view, we should reduce it to some plain statements about what is and is not a reliable source, supplemented by further plain statements identifying a hierarchy of source categories (academic, media, etc). If, when a given source exists ("reliably") in more than one of those categories, editors are indeed required to choose one rather than another, the hierarchy should make that plain. Language such as "usually ... the most reliable ... where available ... but they are not the only ... may also be used ... particularly if" is not the stuff of policy and should be relegated to the related guideline, WP:RS (discussed below). PL290 (talk) 08:17, 14 September 2010 (UTC)

A hierarchy would allow editors to exclude sources lower down for that reason alone, and that will not always be done for good reasons. There's a constant attempt on Wikipedia to elevate scholarly sources over non-scholarly ones, and while out of context that seems supremely sensible, in context you too often see it done to remove criticism of the way the scholarly sources are approaching things. Or to highlight issues the scholarly sources don't want to highlight, or are too slow to deal with.

Practically speaking, it would be impossible to create a hierarchy that would make sense. What would count as academic? Someone with a job in a top university, any university, any college or seminar? Someone who used to have such a job, someone who's never had one but is read by academics? Peer-reviewed well, peer-reviewed badly, non-peer-reviewed? Someone working in a university but published by a non-academic publisher? Someone not working in a university but published by a top academic publisher? There's no magic line between good and bad sources along academic lines, because there are good and bad academics.

Source choice boils down to common sense, not wanting to push a POV, and above all intellectual honesty, none of which we can legislate for. All this policy can do is present what we mean by reliable, meaning "good enough," and hope that intellectual honesty wins the day. SlimVirgin ^{talk|contribs} 08:45, 14 September 2010 (UTC)

I knew the word hierarchy would not be popular (I don't advocate it myself) but that is precisely what the current wording (imprecisely) achieves: Academic publications, such as peer-reviewed journals and books published by well-regarded academic presses, are usually the most reliable and most valuable sources in the topic areas where they are available. However, they are not the only reliable sources in such areas. Non-academic sources may also be used, That wording has established that non-academic sources are second-rate: A is usually best, but B may also be used, therefore B is lower in the hierarchy. PL290 (talk) 09:06, 14 September 2010 (UTC)

Yes, it's unfortunate that it's worded that way, but at least we do make clear that non-academic sources are reliable too in the same areas. I'm arguing that we must not write anything else that entrenches a hierarchical view. Otherwise we end up with the kind of situation we saw in the climate change articles: relatively junior academics being regarded as acceptable, with writers for the New York Times and BBC being rejected. And I mention that only as an example, not to make a political point. The same thing is happening in lots of areas. SlimVirgin ^{talk|contribs} 09:15, 14 September 2010 (UTC)

I agree: see new proposal 3 below. PL290 (talk) 10:22, 14 September 2010 (UTC)

I don't understand what you mean by "relatively junior academics". NYT and BBC have the better science writers, but even they get things wrong. I think that especially in global warming related science, we need to stick to peer-reviewed sources to make sure that exaggerated claims and misunderstandings stay out of the articles. Awickert (talk) 15:53, 16 September 2010 (UTC)

(outdent) I actually like the idea of a hierarchy, because of how it can work in a logical flow-chart-like manner. Consider the following scenarios:

• Someone sees a news article on science or medicine and writes what they read in an article. Someone else looks at those changes and gets the original paper. If the paper and the news article agree, they can use the paper to further expand the article. If they disagree, then the news article is removed (per game of telephone), and the information in the scientific article replaces it. I think that this kind of valuing of articles will help keep up the factual integrity of WP. In other words, non-scientific sources are reliable for scientific topics until they are found to contradict the original sources that they supposedly represent.
• How about a story about people's feelings about the science? Here, now, the news sources are primary pieces of information, as the scientific articles have no bearing on this.
• How about non-scientific sources criticizing scientists? I would disagree with SlimVirgin here and say that these are not acceptable. Especially in climate change, a lot of criticisms have are based in nothing but a misunderstanding of the basic mechanisms that control climate. Scientific journals have the comment-and-response process in which others can write in with their concerns about a particular article, providing a peer-reviewed high-quality document that refutes a paper. Until this happens, there is no good reason to believe that (especially in a political hot-button area), criticisms of the science are reasonable. Awickert (talk) 16:11, 16 September 2010 (UTC)

Hope I haven't just added too much onto a stale thread, but I was just told about this page, and wanted to add my 2 cents. Awickert (talk) 16:11, 16 September 2010 (UTC)

Consistency with related guideline

The "equivalent" paragraph at WP:RS currently says:

Many Wikipedia articles rely on scholarly material. Academic and peer-reviewed publications are usually the most reliable sources when available. However, some scholarly material may be outdated, superseded by more recent research, in competition with alternate theories, or controversial within the relevant field. Try to cite scholarly consensus when available. Reliable non-academic sources may also be used in articles about scholarly issues, particularly material from high-quality mainstream publications. Deciding which sources are appropriate depends on context. Material should be attributed in-text where sources disagree.

Quite a difference. Food for thought as we consider what the right wording is, and also about the wisdom of duplicating the same level of detail in more than one place. PL290 (talk) 16:54, 13 September 2010 (UTC) PL290 (talk) 16:54, 13 September 2010 (UTC)

(edit conflict) Let's add the second half of WP:RS, for reference:

Mainstream news sources are generally considered to be reliable. However, it is understood that even the most reputable news outlets occasionally contain errors. Whether a specific news story is reliable for a specific fact or statement in a Wikipedia article is something that must be assessed on a case by case basis. When using news sources, care should be taken to distinguish opinion columns from news reporting.

* For information about academic topics, it is better to rely on scholarly sources and high-quality non-scholarly sources. News reports may be acceptable depending on the information in question; as always, consider the context.

* While the reporting of rumors has a news value, Wikipedia is an encyclopedia and should only include information verified by reliable sources. Wikipedia is not the place for passing along gossip and rumors.

* Some news organizations have used Wikipedia articles as a source for their work. Editors should therefore beware of circular sourcing.[2]

Let's also bear in mind that this present page is policy, while WP:RS is a guideline that is supposed to follow WP:V policy; where the two differ, it is WP:RS that should be made consistent with WP:V. Even so, the current version of WP:RS is more to the point than what we have here in WP:V. --JN466 19:26, 13 September 2010 (UTC)

Since V governs RS, I'd say we should decide changes here and bring the latter into compliance if necessary. RJC ^Talk_Contribs 19:20, 13 September 2010 (UTC)

Proposal 3

The paragraph we are discussing needs to be considered in the context of points already made by the section in which it appears. This proposal reproduces the whole section, giving markup to show the change to this paragraph. I suggest that the main points are already made by the preceding paragraphs.

Reliable sources

The word "source" as used in Wikipedia has three meanings: the piece of work itself (a document, article, paper, or book), the creator of the work (for example, the writer), and the publisher of the work (for example, The New York Times). All three can affect reliability.

Articles should be based on reliable, third-party (independent), published sources with a reputation for fact-checking and accuracy; this avoids plagiarism, copyright violations, and unverifiable claims being added to articles. Sources should directly support the material as it is presented in an article, and should be appropriate to the claims made.

The appropriateness of any source depends on the context. In general, the best sources have a professional structure in place for checking or analyzing facts, legal issues, evidence, and arguments; as a rule of thumb, the greater the degree of scrutiny given to these issues, the more reliable the source. Electronic media may also be used, subject to the same criteria.

Academic and peer-reviewed publications are usually the most reliable sources where available, such as in history, medicine, and science, but they are not the only reliable sources in such areas. Material from reliable non-academic sources may also be used, particularly if it appears in respected mainstream publications. Other reliable sources include university-level textbooks, books published by respected publishing houses, magazines, journals, and mainstream newspapers. Electronic media may also be used, subject to the same criteria.

When identifying a reliable source for a topic area such as history, medicine, and science, do not assume that scholarly material is necessarily suitable, since it may be outdated, superseded by more recent research, in competition with alternate theories, or controversial within the relevant field.

Self-published expert sources are regarded as reliable in limited circumstances [...]

PL290 (talk) 10:22, 14 September 2010 (UTC)

• No, this does not work. Whether a publication is electronic or not is of no concern to this policy; the key is whether the source is published, in the sense that it is not self-published. If a source is superseded by more recent research, then that does not mean it is not irrelevant, or cannot be used. This proposal is entirely misleading. --Gavin Collins (talk|contribs) 10:24, 14 September 2010 (UTC)
• Nope. Some fields have a long tail (history). Some fields have a short tail (medicine). Current or Proposal 2 have better policy level content. Fifelfoo (talk) 10:29, 14 September 2010 (UTC)
• I must be misreading, because this proposal doesn't work at all. Please clarify-- it appears to completely downgrade MEDRS and encourage lay media for bio/med articles. SandyGeorgia (Talk) 10:50, 14 September 2010 (UTC)
• Absolutely not. It's in complete contradiction with the spirit of the proposed change. Fvasconcellos (t·c) 12:15, 14 September 2010 (UTC)
• Non-starter. Electronic vs paper only matters in the sense that they need different types of archives. Even the first line is factually incorrect: The New York Times is not a publisher but a newspaper, from the similarly named publisher The New York Times Company. The nub of value in this line however is that we need to distinguish the article from the publication and the author for purposes of V and RS. I would contend that it is the article which must be available for verification and which must be reliable for the support of the assertion against which it is cited. LeadSongDog come howl! 18:16, 15 September 2010 (UTC)

Proposal 4

Maybe we are trying to do too much here. Here is an alternative proposal, keeping it simple:

Policy

Proposal

Academic and peer-reviewed publications are usually the most reliable sources where available, such as in history, medicine, and science, but they are not the only reliable sources in such areas. Material from reliable non-academic sources may also be used, particularly if it appears in respected mainstream publications. Other reliable sources include university-level textbooks, books published by respected publishing houses, magazines, journals, and mainstream newspapers. Electronic media may also be used, subject to the same criteria.

Academic publications, such as peer-reviewed journals and books published by well-regarded academic presses, are usually the most reliable sources in the topic areas where they are available. Non-academic sources may be used as well, particularly if they are respected mainstream publications. This includes books published by reputable publishing houses as well as mainstream newspapers, magazines, and journals. Electronic media may also be used, subject to the same criteria.

This is comparatively bland, but keeps it general. It allows guidelines like WP:MEDRS latitude to address the finer points, and also works well for those topic areas that do not have an academic literature devoted to it. --JN466 13:31, 14 September 2010 (UTC)

Getting better, but I still don't like the fact that specific reference to well-developed guideline pages (medicine and science) is dropped. This wording gives us no idea when we might use non-academic sources, as spelled out in MEDRS, or when we might use media sources for contemporary politicians. Waiting for additional feedback ... SandyGeorgia (Talk) 13:38, 14 September 2010 (UTC)

Good. :) Where do you see a specific reference to WP:MEDRS in the present policy? The mere mention of "medicine" as an example of a topic that has academic coverage does not indicate to the reader that there is a content guideline for that topic area. The only guideline linked to in the present policy is WP:IRS, and that then has a link to WP:MEDRS. In terms of the hierarchical structure of our policy–guideline system, that makes sense to me. --JN466 14:01, 14 September 2010 (UTC)

More detailed guidance of the kind you have in mind for Chávez-type situations and medical articles is probably better housed in WP:IRS. This can better differentiate between situations where press articles are not so good as sources (medicine) vs. situations where press articles are vital sources for covering current affairs (Chávez, climate change). To some extent, WP:IRS does this already, but we might want to do some fine-tuning in WP:NEWSORG. --JN466 14:22, 14 September 2010 (UTC)

Too little improvement. This still carries an implicit bias against e-form journals and books that is hard to justify. It also still unjustifiably implies that all categories of articles (systematic review, literature review, original research, reader comments, author responses, letters, obituaries) are of equal reliability, and it still attaches more significance to the publishing house than the imprint or editorial board. Considering the broad spectrum of imprint quality within individual publishers this makes no sense at all. LeadSongDog come howl! 17:57, 15 September 2010 (UTC)

I don't follow. In which part of the wording do you see a bias against books and e-form journals, either in the proposal, or in the present policy text? --JN466 18:16, 16 September 2010 (UTC)

"...may also be used..." implies (at least to me) that it is a lesser option. "...Academic works, published in either paper or electronic form, ..." would avoid that.LeadSongDog come howl! 18:29, 16 September 2010 (UTC)

Thanks for clarifying, now I see. I've always assumed that sentence about electronic media refers to websites like http://www.music-news.com and such. If a peer-reviewed journal is only available in electronic format (I can think of at least one example, which is highly regarded in its field), then it is still a peer-reviewed journal in my book. However, if an academic wrote something on his blog, or put up an unpublished paper on his website, then I'm not so sure Wikipedia would want it used as a source (it would be a WP:SPS). What examples did you have in mind? --JN466 19:25, 16 September 2010 (UTC)

Well, I tend to come at these things from a hard-science perspective, but musicology too has a formal academic side. The point is that whether I pick up a printed paper journal article or download a scan of it from a trusted archive doesn't change the reliability of that article. Publishers know that their readers know this and derive more and more of their revenue from providing that trusted archive themselves, but libraries and others do it too. I can get the Journal of Musicology or The Journal of the American Musicological Society either in e-form from JSTOR or on paper by walking into a university library. I don't need to rely on Rolling Stone or Billboard for formal semiotic analysis. LeadSongDog come howl! 18:46, 20 September 2010 (UTC)

We need to stress that, in principle, only reputable peer reviewed scientific journals are acceptable as sources. When such sources are not available (e.g. in case of breaking news; despite Wikipedia usually not bringing the latest news per WP:NOTNEWS, sometimes we still want to write about a news story), then we can use other sources, but the information must not be fundamentally in conflict with what can be distilled from peer reviewed sources.

E.g. when the CRU hacking incident story broke, one obviously had to write something about climate science using newspapers as sources. It is then important to filter out statements from those sources that are in conflict with established science that could not in any way have been affected by that incident (i.e. assuming that the scientists had falsified data, which later turned out not to be the case). Then after a while, scientific journals will retract articles once it is clear that the results are bogus. If that doesn't happen while some newspapers still suggest that the results may be bogus, we cannot consider the newspaper stories to be reliable, unless there is evidence that the editorial standards of the journals may have also been compromized (e.g. if many scientists accused of fraud also happen to be the editors in chief of the leading journals). Count Iblis (talk) 03:00, 16 September 2010 (UTC)

What is to be taken as a "respected mainstream publication". I'm sure many would take the New York Times as an example, but it is only respected as a newspaper (respected for reporting on events as they are understood at the moment, for example, Sadam's weapons of mass destruction), not as a source of (for example) scientific opinion or fact. These vague descriptions are going to lead to unending debate over what they really mean on Talk pages. Brews ohare (talk) 15:38, 16 September 2010 (UTC)

Agree. We must distinguish science and medicine from other areas, even with the NYT, which is often dramtically wrong in the field of medicine, and at odds with peer-reviewed secondary medical sources but reliable on more contemporary issues like Chavez. Perhaps this means we need to specifically link to the medicine and science guideline pages, which were developed based on broad consensus across Wiki, and posted to every forum we could think of (in the case of medicine) as they were being developed. SandyGeorgia (Talk) 16:04, 16 September 2010 (UTC)

Count Iblis, what about a topic like Transcendental Meditation? TM practitioners have produced literally thousands of peer-reviewed journal articles investigating the benefits of TM. They far eclipse in number the studies of TM that have been published by outsiders, some of whom, I believe, have criticized the integrity of this body of research. Are you saying that the pool of peer-reviewed literature should suffice to produce NPOV coverage of TM? This would exclude books written by religious scholars, published by university presses, and it would exclude the viewpoints of the entire press. What about notable figures in art and literature, and their works? I agree that too little use is made of peer-reviewed studies in this field in Wikipedia, but is it reasonable to exclude newspaper content like book reviews, author interviews etc. as sources, just because there is a wealth of peer-reviewed studies on a prominent author or artist? --JN466 17:46, 16 September 2010 (UTC)

"Peer reviewed" isn't the only factor in a field like TM: almost anyone can get anything published. The question is whether results of primary studies are reviewed by secondary sources, which may reveal problems with primary studies, even if published in peer-reviewed journals. If a controversy is significantly broad to have been covered in the laypress, we can use media sources responsibly, but primary studies unreviewed by secondary sources in a field of TM is precisely what we should avoid, and the laypress frequently gets it wrong when analyzing and reporting the science. Google scholar is somewhat useless for locating quality sources (it even includes Wikipedia), but PubMed lists 283 secondary reviews of TM, so it seems unlikely we need to rely on the laypress. SandyGeorgia (Talk) 18:16, 16 September 2010 (UTC)

In our present article on TM, the fees for TM training for example are all sourced to newspapers. --JN466 18:28, 16 September 2010 (UTC)

That's not "medical" info-- it is the kind of info that can be sourced to other reliable sources (unless peer-reviewed sources are also available, in which case, they should be used). SandyGeorgia (Talk) 18:32, 16 September 2010 (UTC)

Indeed. I still think that Hans Adler's comment above, 14:11, 13 September 2010, outlines a very sensible basic approach. I'd welcome suggestions on how we can translate that into a policy wording. --JN466 19:05, 16 September 2010 (UTC)

Sandy, Wikipedia:Scientific citation guidelines, which you linked to earlier as an example of a content-specific sourcing guideline along with WP:MEDRS, does not really help editors identify the most reliable sources in science, the way WP:MEDRS does for medicine. WP:SCG is concerned with citation formatting, not with identifying reliable sources. --JN466 19:16, 16 September 2010 (UTC)

OK, thanks (I was part of developing MEDRS, but not SCG); I added MEDRS to See also, but removed SCG. SandyGeorgia (Talk) 19:19, 16 September 2010 (UTC)

Ok. By the way, good job on MEDRS. --JN466 19:29, 16 September 2010 (UTC)

I agree that fringe/pseudo science subjects are exceptional cases, especially when we are not dealing with the "hard sciences". Secondary reviews as SandyGeorgia points out can be used, but I would say that this has to be done with care. One has to demand that those secondary reviews are not inconsistent with established science, which in turn is what one can distill from peer reviewed sources. So, the secondary source may invoke basic physcis to point out that a peer reviewed study on TM is bogus. Then I should be able to trace back those physics arguments to the peer reviewed physics literature and verify that the debunking done in the secondary non-scientific source is correct. Of course, we don't have to demand that one actually do this literature reseach. What matters is that we trust the secondary source enough that we can skip this step. But the moment we've doubts about this, we should not use that secondary source. Count Iblis (talk) 20:13, 16 September 2010 (UTC)

Another factor worth keeping in mind is that other topic areas – in the humanities for example – may not have secondary reviews and meta-studies of the kind common in medicine. In medicine, many peer-reviewed journal articles report the result of statistically designed studies testing a hypothesis (e.g. does the treatment have a statistically significant effect, or not). Peer-reviewed journal articles on music, art or literature, for example, do not have that kind of structure, and do not lend themselves to meta-analyses in this way. Whatever wording we have in this policy has to work for all our topic areas. --JN466 21:23, 16 September 2010 (UTC)

Myspace?

Several editors have added Category:Jewish Poker Players to Barry Shulman and have cited his MySpace page as a source. What do you think? Does this pass under WP:SELFPUB?--TM 15:06, 19 September 2010 (UTC)

Can we be 100% certain that the Barry Shulman who created the Myspace page is the same Barry Shulman that is the subject of our article? If so, then it would pass WP:SELFPUB... if not, then no. Blueboar (talk) 18:19, 19 September 2010 (UTC)

How can we verify that on Myspace?--TM 17:10, 20 September 2010 (UTC)

I don't know. It may be that this is not possible. All I am saying is that it would need to be done. Blueboar (talk) 17:19, 20 September 2010 (UTC)