Wikipedia talk:Identifying reliable sources (medicine)

Uptodate.com

There has been mention previously about the us of Uptodate.com as a secondary source. Wondering what people thought are about structuring references to this source? I have started siting both Uptodate and the primary research article so people without access can look at the primary source.

Doc James (talk) 15:31, 16 July 2008 (UTC)

UpToDate is a good source, but the best sections on UpToDate heavily reference the actual medical literature. As much as possible, we should use UpToDate as a guide to which literature is relevant and how to weight it,but citing UpToDate directly is a bit iffier since it's essentially one expert's opinion. That said, the site has an excellent reputation for accuracy and is a widely used professional resource, so it's not a matter of accuracy, more of how we approach things. MastCell ^Talk 15:55, 16 July 2008 (UTC)

I think UpToDate is a good resource, but heavily restricted even to many editors here, making it hard to access.

Jmh649 has started this discussion specifically because UpToDate is one of the secondary sources that cites PMID 814892. Now if we could find another secondary source that cites that paper, then I think this source would be preferable to UpToDate. JFW | T@lk 17:16, 16 July 2008 (UTC)

Or you could just use that primary source in good faith, and refer anyone who questions you to uptodate.

Should we set up a template, like for {{eMedicine}} ? --Steven Fruitsmaak (Reply) 17:25, 16 July 2008 (UTC)

Just one data point. I'd never looked into uptodate.com, so I thought I'd check it by looking at its coverage of the epidemiology of autism, an area I've worked on in Wikipedia. Uptodate.com's first sentence on the subject is "The prevalence of ASD in the United States and other countries has increased since the 1970s and particularly since the late 1990s"[1]. But this claim isn't true, and the source they cite (Williams et al. 2006, PMID 15863467) doesn't make the claim. What the source says is that the reported prevalence has increased, but it is not known whether the actual prevalence has increased; it could be that the changes are due to changes in diagnostic criteria or other factors. Admittedly this is just one data point, but still, wow, that's a pretty basic error. Accordingly:

• I'd like to second MastCell's suggestion to use uptodate.com as a source of plausible citations rather than to cite uptodate.com directly.
• If there is some dispute about whether to cite PMID 814892 (a 30-year-old letter to the editor?) I wouldn't rely on uptodate.com to resolve it.

Eubulides (talk) 17:46, 16 July 2008 (UTC)

Following up on my own comment: I see that later text in uptodate.com does make the point that autism's bureaucratic prevalence is not the same as its true prevalence, so I was too critical: it's more a major editorial error in uptodate.com than an actual content error. And it is just one data point. Still…. Eubulides (talk) 17:51, 16 July 2008 (UTC)

That actual reference from Uptodate JFW is referring to is: Lee L, Kumar S, Leong LC (1994). "The impact of five-month basic military training on the body weight and body fat of 197 moderately to severely obese Singaporean males aged 17 to 19 years". Int. J. Obes. Relat. Metab. Disord. 18 (2): 105–9. PMID 8148923. {{cite journal}}: Unknown parameter |month= ignored (help)CS1 maint: multiple names: authors list (link) Doc James (talk) 20:31, 16 July 2008 (UTC)

Mea culpa, dropped a digit. I have refactored your comment to include the citation rather than the entire abstract. A primary research study in 197 subjects remains problematic. JFW | T@lk 21:12, 16 July 2008 (UTC)

In follow up I would like to mention that the article from Uptodate was used to emphasis a point made by the systematic review. That the amount of exercise has an effect on the degree of weight lose. It was not used to "debunk" the systematic review. See page on obesity.

Doc James (talk) 15:25, 17 July 2008 (UTC)

uptodate is peer reviewed - here is what they say about there process:

Editorial policy

Authors

All topics in UpToDate are written by the listed authors in conjunction with a deputy editor. Authors are identified as experts by the Editors-in-Chief, our editorial staff, and the participating societies. Exceptions are guidelines from major societies, which are added to UpToDate in their original form. All material is originally prepared by the contributing author(s) whose name(s) and affiliation(s) appear in the upper left corner of each topic. This material is reviewed extensively by our physician editors and peer reviewers for accuracy and completeness of the literature search, and for consistency with all aspects of the editorial policy.

Doc James (talk) 17:53, 20 July 2008 (UTC)

There is no doubt that UpToDate is peer-reviewed, but it remains expert opinion in review form with an uncertain selection process. The content is also not permanent and the same URL may lead to content that is liable to change. If there is a recent review available in a "real" journal this is vastly preferable over UpToDate.

What does the panel think about using UpToDate as a "hidden" supportive source. For instance, a primary research study is cited on UpToDate, thereby confirming its relevance, so the editor who includes that reference makes a comment in the reference code (not visible to the general reader) that the reference is cited by UpToDate? Thereby we satisfy the demands of this policy without overburdening the reader with a resource they are unlikely to be able to access even by interlibrary loan? JFW | T@lk 07:47, 21 July 2008 (UTC)

Hidden sources give me the willies. Wikipedia should be open about its sources, even if the sources themselves are closed. Eubulides (talk) 07:58, 21 July 2008 (UTC)

I don't understand how this is really "hidden". We are actually citing the primary source, and justify its importance under the "reviews preferred" aspect of WP:MEDRS by saying that it was cited by UpToDate. So long as you then actually read the primary source, to meet WP:CITE#SAYWHEREYOUGOTIT, then I have no problem with this. WhatamIdoing (talk) 20:48, 22 July 2008 (UTC)

Sources the ordinary reader can't see are hidden to the reader. If there's a reliable review, and the reason we're citing the primary source is that the reliable review cited it, then we typically should be citing the reliable review and not the primary source. This is true regardless of whether the review is published in uptodate.com or elsewhere. There are some exceptions to this (for example, notable primary sources worthy of discussion directly in the article text), but the exceptions shouldn't be common; and even then, it's better to cite recent reviews in addition to citing the primary source directly. Eubulides (talk) 23:03, 22 July 2008 (UTC)

RSN that may interest some editors here

I have revived a question at RSN on pseudonymous authors of "letters to the editor" in otherwise reliable sources. It may interest some editors here, and as the only editor who supports this source is demanding responses from additional editors to demonstrate "consensus", I am looking for editors who are willing to read it and respond.

Note that I don't have a vested interest in the outcome; I just want it very thoroughly settled one way or the other. WhatamIdoing (talk) 18:06, 4 August 2008 (UTC)

Process for guideline status?

Does anyone know what the proper procedure is to have this adopted as a guideline? WhatamIdoing (talk) 17:27, 5 August 2008 (UTC)

Huge can of worms and pet peeve extraordinaire alert. There isn't one, there should be, there is no coordination, but some editors are vehemently opposed to any efforts to centralize and coordinate MoS. In terms of history, we (MEDMOS) were actually the only WikiProject to be subjected to a process. In the olden days, Radiant! tagged our page (historical, I think), and we had to jump through hoops to get added to MoS. We posted to several dozen other Projects, the Village Pump, etc. to garner widespread consensus for our guidelines. To my knowledge, no other MoS page has ever had to do that. When I proposed something similar at MoS, it was rejected. Summary: we have a mess at MoS where a few editors can put together a page and call it a guideline with no mechanism for gathering broad consensus. And I'm supposed to make sure FAs are MoS-compliant, per WP:WIAFA crit 2, even though guidelines often contradict each other. Back when I proposed this, someone started a MoS WikiProject, which was promptly ignored. I could find all of these links if I had to, but it would take me a very long time. Ugh. SandyGeorgia (Talk) 17:32, 5 August 2008 (UTC)

By the way, the place to get our reliable sources text included would be at WP:RS, but it was already moved out of there, back during the old WP:ATT debacle, so that is yet another can of worms alert. SandyGeorgia (Talk) 17:34, 5 August 2008 (UTC)

I'm sorry things are such a mess. So, Sandy, what would make you (and therefore the FA process) happy? Would you like me to declare this a guideline on the grounds that the editors of this page largely agree that other people ought to follow our excellent advice? Would you like me to pretend that such a thought never crossed my mind? Would you like me to invent an "RfG" process for you? WhatamIdoing (talk) 17:51, 5 August 2008 (UTC)

Should we make this a guideline?

I find the arguements for making this a guideline compelling. Whilst some sensible concerns raised, overall the WP:MEDRS acts as a additional help to the main WP:RS, giving pointers & issues that may apply in biomedical topics. Of course any specific review article might be found wanting, this can also occassionally apply to primary source research papers which may be questioned with authors & journals later ammending or even rarely retracting. Clearly therefore one can not be absolutist, and so this is a guideline and not a policy with header instruction that it be "treated with common sense and the occasional exception". In general therefore RS prefers secondary sources to primary in outlining a topic, but MEDRS seeks to give a little greater creadance and advice on citing the primary sources for the origin of items of knowledge.

As RS states "In science, single studies usually are considered tentative evidence that can change in the light of further scientific research. How reliable a single study is considered depends on the field, with studies relating to very complex and not entirely-understood fields, such as medicine, being less definitive. If single studies in such fields are used, care should be taken to respect their limits, and not to give undue weight to their results. Meta-analysis and systematic reviews, which combine the results of multiple studies, are preferred (where they exist)."

I see no direct objection above that common sense is required in looking at how any one source (review article, primary research or indeed a "standard" textbook) compares to the bulk of other sources - for WP:UNDUE weight needs to be considered for views that do not follow the majority viewpoint if we are to maintain WP:NPOV (in essence a review article or text book that presents a view not held by the majority is in a minority whatever type of source it is). That said, a citable-reference in a reliable-publication-source that tries to summarise an overall topic and its current research-base, is likely to assesss the importance or otherwise of individual primary sources better than any editor (who so summarising themselves would be at risk of WP:Synthesis which is original research).

Therefore I promote this MEDRS from proposal to an accepted guideline in light of satisfactory consensus above. This does not of course preclude the guidence to be discussed below and, hopefully, improved further :-) David Ruben ^Talk 00:53, 1 September 2008 (UTC)

As an editor, I have had a 1st go at adding a {{nutshell}} header - feel free to discuss and tweak that David Ruben ^Talk 00:53, 1 September 2008 (UTC)

Thanks David. I think this is a major step forward. I hope we can settle some final disagreements (below) pretty soon now. JFW | T@lk 05:26, 1 September 2008 (UTC)

Systematic reviews and primary sources

This edit removed the following sentence from Wikipedia:WikiProject Medicine/Reliable sources #Article type without discussion:

A systematic review is both a primary and secondary source—it summarizes other papers but it does so in order to research the field and possibly come to a novel conclusion.

with the comment "this is really not true -- a good systematic review basically summarizes previous findings only. Looser reviews do the more primary work." Possibly-controversial edits like this should be discussed on the talk page first, so I took the liberty of reverting the change and moving the text here, for discussion. Eubulides (talk) 02:20, 15 August 2008 (UTC)

I actually support removal of this sentence. In the vast majority of cases, systematic reviews make no new observations at all. Non-systematic (or "narrative") reviews occasionally reflect expert experience without an evidence base, but they remain reviews in their editorial sense. JFW | T@lk 02:22, 15 August 2008 (UTC)

I honestly didn't think that removing this sentence could possibly be controversial, as it seemed to be superflous in addition to being false. I thought about simply cutting the word systematic from the sentence, but I figured that might be controversial! II | (t - c) 02:42, 15 August 2008 (UTC)

I also support the removal of that sentence since it differs from the definition of the term in the article at systematic review and from the use of the term in the lead paragraph of this page. --Jack-A-Roe (talk) 02:57, 15 August 2008 (UTC)

The sentence in question has been present in nearly its current form since the talk page was created in 2006. It was added in the very first change to the article, by Colin, the original author of the page, here. I'll leave a note on Colin's talk page to ask him to weigh in on this issue. Eubulides (talk) 05:31, 15 August 2008 (UTC)

Feel free to correct my mistakes -- I'm just an amateur. I bow to the folk with training in this subject! I wrote that a while ago and might have worded it differently now. I've always hoped, in fact, that someone with training/qualifications in this area would rewrite much of MEDRS, while keeping those aspects that are unique to WP. Re-reading Greenhalgh's How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses), I can see how I thought the "novel conclusion = primary" might occur with such papers. If you guys think that is rare then clearly the sentence isn't useful. Here's my impression: A narrative review by an expert might be likely to have a conclusion that the expert already "knew" before he even started working on the paper. He's set out to describe rather than systematically discover the current situation. A systematic review is an objective exercise in examining the published research and the results might surprise the author(s). Such a review might gain access to the raw data from published trials, and other less accessible input (see the "Checklist of data sources for a systematic review" in Greenhalgh's paper). If the review contains a meta-analysis or other synthesis of results, then arguably it produces its own conclusion (even if this is similar to and adds weight to what was "known" already). The publication of a Cochrane review can make the newspapers, so it must be saying something we haven't heard before or with as much conviction. Still, I agree that it may be simpler to just consider such reviews to be excellent secondary sources. Colin°^Talk 18:33, 15 August 2008 (UTC)

Thanks for following up. In rereading the article it seems clear to me that something needs to be said here about systematic reviews, as systematic reviews are mentioned in the lead and therefore need to be in the body as well. How about if we remove the phrase about primary and secondary source, along with a newly-redundant phrase, as follows:

A systematic review is both a primary and secondary source—it summarizes other papers but it does so in order to research the field and possibly come to a novel conclusion.

Eubulides (talk) 20:09, 15 August 2008 (UTC)

Eubulides, edit away and expand if you can. Please, the more you can add to this proto-guideline the better IMO. Colin°^Talk 20:21, 15 August 2008 (UTC)

OK, thanks. For now the consensus seems to be that this change is OK, so I put it in. Expanding will take more work.... Eubulides (talk) 18:57, 22 August 2008 (UTC)

I support the direction you seem to be taking with your edits above. In my mind systematic reviews are not given their due. They should not be lumped in with primary sources, or with summary reviews of the literature that frequently precede the presenting of original research in a journal article. The latter is often self-selected to support the hypothesis the researcher is evaluating, but I frequently see these reviews (often in abstract form) used as a secondary source. —Mattisse (Talk) 20:32, 15 August 2008 (UTC)

Hmm. I don't oppose the addition of that sentence, but I think it is bit superflous. Systematic review is wikilinked in the lead. We don't really need define it in MEDRS. Incidentally, Mattisse, I actually previously supported emphasizing systematic reviews, but if I recall correctly JFW wasn't really happy about that(not sure what Eubulides thoughts on the matter were). Then again, the page on systematic review already states that these are the standard in evidence-based medicine, which works just as well. Actual medicine guidelines > MEDRS in my mind. I actually think modularity in these articles is best -- keep the things which are strictly MEDRS here, and fork off tangents and definitions elsewhere. And, no offense, but that sentence is a "duh" sentence. It's a "duh" statement that in most cases, reviews will synthesize the literature and come to their own, possibly novel, probably not novel, conclusion. If you want to add that sentence, you're going to have to at least strike the word systematic from it, Eubulides. II | (t - c) 19:07, 16 August 2008 (UTC)

I have never opposed the use of systematic reviews, and indeed I feel that we should place much emphasis on them. But they tend to be relatively narrow in scope and broad in methodological thoroughness. Hence, "narrative reviews" are great for broad sketches, while systematic reviews are better at answering specific questions. JFW | T@lk 21:56, 17 August 2008 (UTC)

• There seems to be some confusion here, as the proposal is to remove some text, not to add it.
• Removing the entire systematic-review sentence would be unwise, as the containing paragraph discusses all sorts of article types, and discussing other article types while omitting mention of systematic reviews (a preferred source) would be very odd.
• It's not a "duh" statement to define a systematic review; we can't expect all Wikipedia editors to know what it is. The paragraph already defines general reviews, and it's useful for it to make a point about systematic reviews and what makes them different. If you prefer a different definition for "systematic review", please propose something.

Eubulides (talk) 23:12, 16 August 2008 (UTC)

I don't think it needs to be defined, and I don't think they come to novel conclusions. Generally a systematic review will have one of two conclusions: 1) Not enough evidence, 2) Enough evidence, some consistent result. The paper pointed by Colin has good summary of them, and I think it would be nice to actually reference that paper in this article. A good definition would note that a systematic review uses a predefined, replicable methodology to select all articles from a certain time period, with predefined inclusion criteria. II | (t - c) 01:15, 18 August 2008 (UTC)

WP:MEDRS #Article type defines/discusses general reviews (which are mentioned in the lead); for consistency shouldn't it also define/discuss systematic reviews (also mentioned in the lead)? It would be odd for the lead to mention a topic not discussed in the body. The current (and proposed) wording doesn't say that systematic reviews always come to novel conclusions, only that they "possibly" come to novel conclusions. Again, if you could propose specific wording to replace the proposal above, that would help drive the discussion forward more quickly. Eubulides (talk) 19:42, 18 August 2008 (UTC)

Although general (journalistic or narrative) reviews are a good introduction to a topic, they should be used with a consideration of the sources cited by the author. When citing information on the effectiveness of certain therapies, consider looking for a systematic review. [Alternative: Systematic reviews are the considered to be the gold standard in evidence-based medicine in regard to complex medical questions.] A systematic review uses a predefined, replicable methodology to select all articles from a certain time period in order to answer a specific question. This it thought to reduce bias and increase thoroughness, since articles which do not support the researcher's preconceived conclusion must be included if they fit the inclusion criteria.

— [2]

II | (t - c) 20:06, 18 August 2008 (UTC)

I assume this is intended to replace the last 3 sentences of WP:MEDRS #Article type? If so, the proposed replacement has two major problems. First, it disses general reviews far too much. General reviews, when done well, are better sources for Wikipedia than systematic reviews are, since they match Wikipedia's encyclopedic aims better. Second, WP:MEDRS #Article type is descriptive: it does not (and is not intended to) give advice. Advice is in WP:MEDRS #Some definitions and basics and proposed changes to advice should be made there. Eubulides (talk) 20:13, 18 August 2008 (UTC)

Sorry, it is intended to replace the last sentence only. I tweaked it slightly. General reviews are only as good as the sources (or perhaps the credentials) of the author. If the review cites only one primary study for a claim, its reliability is not much better than that primary study. I don't understand how general reviews are more encyclopedic. They are certainly not more reliable than systematic reviews. As far as "advice", I think that's your opinion. This page is a guide, and I think it should read as such. But if you prefer, the second sentence can be changed to "" Any other advice? II | (t - c) 20:28, 18 August 2008 (UTC)

(Undent.) As discussed previously, editorial judgement is advised when selecting sources and citing their contents. But secondary sources will generally trump primary sources for most citations, and I don't think we should try to codify for exceptions. I dispute the claim that systematic reviews can answer complex medical questions; in the vast majority of cases, the data is so insufficient or contradictory as to allow for the drawing of definite conclusions about complex medical questions. Back to you, ImperfectlyInformed. JFW | T@lk 20:50, 18 August 2008 (UTC)

Yes, and here we go again. I'm not codifying for exceptions. I'm just pointing out that some attention should be paid to the sources cited by a review. That's probably the most important part of editorially analyzing reviews. As far as systematic reviews actually answering the questions, perhaps you're right. But they are considered to have the best chance at answering these types of contentious questions, and they are the best at putting the question into proper context, since they attempt to include all context. II | (t - c) 21:17, 18 August 2008 (UTC)

I think we're actually much in agreement regarding the place of various types of publications. In a world full of evidence, systematic reviews (especially of the Cochrane flavour) would answer any question. JFW | T@lk 21:31, 18 August 2008 (UTC)

agreed. Systematic reviews of this sort are in fact the goldstandard and the very model of an authoritative secondary/tertiary source. ; the Cochrane series are not the only ones--there are also other formal consensus statements issued by appropriate groups. When they don'ta gree, which is not unheard of, w need only present the different ones equally. I wish other fields were as well organized for such things. DGG (talk) 04:14, 19 August 2008 (UTC)

I agree with Eubulides and JFW that the proposed text disses general reviews too much. I don't really know what the "should be used with a consideration of the sources cited by the author" is supposed to add. Many editors won't have the ability/experience/training to judge whether the review's sources are balanced/good (and so just have to trust that a review published in a good journal is probably good). Those that are capable of judging the quality of a review probably don't need to take advice on this matter from Wikipedia! I wonder if II you are thinking too much about arguing about the efficacy of therapies rather than trying to write a rounded encyclopaedic article on a subject. A good general review provides far more than just an "introduction to a topic". In the ketogenic diet, I was able to source whole paragraphs to narrative review sources such as Clinical Aspects of the Ketogenic Diet. I used two systematic reviews: Ketogenic Diet for the Treatment of Refractory Epilepsy in Children: A Systematic Review of Efficacy and Ketogenic diet for epilepsy (the former funded by US insurance companies in 2000 and the latter from the Cochrane Database of Systematic Reviews 2003). Both answered only one question, "does it work", and both said "weakly yes, but there are no randomised controlled trials" and are dated since such a high quality trial was finally published in 2008 (PMID 18456557). Both those systematic reviews read much like a primary research paper (background, methods, results, discussion, conclusion) and are so focussed that they aren't any use for the rest of the WP article.

Guidelines

I agree with DGG about the "other formal consensus statements issued by appropriate groups". Currently, MEDRS doesn't mention clinical guidelines. Two excellent sources I have used are the guidelines for Scotland and England and Wales. I know less about the US but I have seen consensus statements begin published (self or in journals) by various medical professional bodies and by some of the more respectable charities. Could we add some mention of this to MEDRS? Colin°^Talk 08:28, 19 August 2008 (UTC)

Guidelines tend to be limited in geographical scope. NICE does not pretend that its guidelines are valid outside England and Wales, and therefore not everything recommended by that body is therefore necessarily representative of practice in the rest of the world. An example would be the risk stratification algorithm for warfarin in people with atrial fibrillation; this is a novel system not used elsewhere. Similarly, its guidelines on computed tomography of the brain after head injury may differ substantially from similar guidelines elsewhere. As a result, I prefer to use reviews in preference over guidelines; that said, they can be excellent sources of information (e.g. the British Society of Gastroenterology guideline at coeliac disease). JFW | T@lk 18:49, 19 August 2008 (UTC)

• I like the idea of mentioning guidelines, but agree with JFW that the area's a bit tricky. In Chiropractic I've had to deal with the problem that there are three competing groups of guidelines for treatments used by chiropractors: one by the majority "mixer" chiropractors, another by the minority "straight" chiropractors, and another being the mainstream medical guidelines. The three groups differ markedly in their results. Even if we restrict ourself to mainstream medical guidelines developed by national governmental organizations, guidelines in different countries disagree markedly; see Murphy et al. 2006 (PMID 16949948).
• My experience is that guidelines tend to be less useful than reviews as sources to Wikipedia articles than reviews are, not necessarily because they are lower quality, but because their goal (advice about treatment) is more distant from Wikipedia's goal (encyclopedic coverage) than the goal of reviews (broad and balanced coverage of a topic). Though of course guidelines often have good reviews within them.

Eubulides (talk) 19:19, 19 August 2008 (UTC)

Guidelines must be carefully evaluated. An institution like the Mayo Clinic may offer "guidelines" meant for the general public without any attempt to source or valid the claims. —Mattisse (Talk) 20:03, 19 August 2008 (UTC)

Again, editorial judgement is advised. The NICE guidelines, too, are sometimes expert/experience based rather than supported by evidence because such evidence is often lacking. Also, Matisse, we are talking about professional guidelines here, adherence to which is often mandatory and a yardstick by which professional performance is meausured, as opposed to resources for the general public that cannot legitimately be called "guidelines" but rather "advice" or "tips". JFW | T@lk 20:19, 19 August 2008 (UTC)

Well, psychology articles are notoriously awful, but these sorts of references are often used in articles on topics in psychology. I would like to see a firm way of discouraging it. —Mattisse (Talk) 20:23, 19 August 2008 (UTC)

Sounds reasonable. JFW | T@lk 20:35, 19 August 2008 (UTC)

I have to agree with matisse, some of the guidelines I have seen put out by otherwise reputable bodies (stocked with professionals) have been pretty poor at times. Maybe a good caveat WRT secondary sources is be especially careful if one finds two or more sec. sources putting forth substantially variant positions/findings/information etc. Cheers, Casliber (talk · contribs) 22:29, 22 August 2008 (UTC)

Funny one I found - in australia (and apparently America), screening tools are fait accompli these days and promoted as such, yet the data seems to tell a different story..actually there was another BMJ paper which was even more cautious. Cheers, Casliber (talk · contribs) 22:34, 22 August 2008 (UTC)

Conflicting advice?

Hi - I apologize for making a few late edits to this proposed guideline, but I trust I'll be reverted if they're out of line. In any case, I wanted to raise what looks like conflicting advice. We emphasize the importance of secondary sources as the basis for medical articles - great. But in the section headed "In science, avoid citing the popular press", we tell people to cite specific researchers' papers rather than newspaper coverage of them. I understand where this is coming from, but we seem to be telling people to go to primary sources in preference to a good popular-press account, which seems a bit iffy. I'd prefer to re-title the section to "Use the popular press with caution" rather than "Avoid the popular press", and perhaps to indicate that popular press can be useful to adduce the sociological and cultural aspects of a disease, drug, etc. For instance, (quality) popular-press accounts are pretty much essential to a full understanding of the context around Lyme disease or Vioxx. "Avoid" just seems to strong. Thoughts? MastCell ^Talk 20:28, 27 August 2008 (UTC)

I'd agree with that. There are sometimes relevant stories covered in the op. press...I just need to think of an example now....maybe a medication taken in a notable court case, or erson diagnosed with a particular condition. etc. Cheers, Casliber (talk · contribs) 20:37, 27 August 2008 (UTC)

I agree also. The indiscriminate use of primary sources (small n, a few case histories, etc.) even from PMID to "prove" a general point is more of a problem, in my opinion, then using reputable popular press references which can be presented for what they are, a popular press review. —Mattisse (Talk) 21:09, 27 August 2008 (UTC)

Press reports are usually unsuitable as secondary sources. Here in the UK every rat study on beta amyloid is reported in the Daily Wail with huge headlines: "Dementia cure round the corner". JFW | T@lk 22:56, 27 August 2008 (UTC)

Well, the Daily Mail is certainly not a great source. Who was it that said they classify every substance known to man as either causing or curing cancer? :) I do think there's a role for high-quality popular press, like Scientific American or New Scientist (though I still haven't forgiven the latter for their inexcusable DCA hype, it's out of character). The New York Times also tends to have decent to excellent science and health coverage. MastCell ^Talk 23:05, 27 August 2008 (UTC)

Agree that whoever was behind the DCA garbage was hopefully and unceremoniously sacked. In other words, it does again boil down to editorial judgement on which popular press source is to be regarded as reliable. JFW | T@lk 23:18, 27 August 2008 (UTC)

It might be useful to list a few examples of legitimate uses for a popular media, such as reports on court cases, as Casliber mentions. Business/marketing issues, information about drug approvals, all sorts of scandals -- there's a lot of not-directly-scientific stuff that goes into some medicine-related articles. WhatamIdoing (talk) 00:14, 29 August 2008 (UTC)

Like MastCell, I'd like to re-title this section. I think the main point to make here is that it's better to cite a study directly (and correctly) than to cite a mass media summary of a study. More links in the chain = more opportunities for errors. Also, quite a lot of what we need in basic medical articles isn't going to be available in a good newspaper. (When was the last time you saw a good anatomy article in The London Times?)

Perhaps more importantly, I think it should be expanded to add one sentence about things that mainstream media does better than scientific journals, such as the non-scientific end of things. It's not really "don't cite the popular press" -- it's really "don't cite the popular press for medical/scientific details. WhatamIdoing (talk) 00:51, 31 August 2008 (UTC)

Popular press and primary sources

This edit altered WP:MEDRS #Some definitions and basics to say that most articles and press releases are primary sources in medicine. My experience is just the opposite: in medicine the primary sources are the research articles, and the popular press and press releases comment on the research articles. I reverted the change, and perhaps we can discuss the situation further here. Eubulides (talk) 01:40, 29 August 2008 (UTC)

Press releases are usually issued by companies or organizations soon after the publication of a scholarly article or even before such an article. They are usually a shortened or popularized version of the scholarly article without analysis, deeper interpretation or review of other relevant sources, which is a necessary pre-requisite for it being a secondary source. For example, these press-releases (like most press releases) are clearly primary sources: [3], [4], [5], [6] Please provide more evidence of your belief that press-releases are secondary sources. Paul Gene (talk) 02:11, 29 August 2008 (UTC)

A press release is a press release: it is what it is. We don't need to say anything specific about them on this page; that is for WP:V and WP:RS. SandyGeorgia (Talk) 02:13, 29 August 2008 (UTC)

Sandy, do you suggest deleting the mention of press releases from the definition of secondary sources? I would support that. Paul Gene (talk) 02:42, 29 August 2008 (UTC)

No, I think we've got it just right as is. We're in line with WP:PSTS and anything beyond that should be dealt with on main policy pages. SandyGeorgia (Talk) 02:51, 29 August 2008 (UTC)

Sandy, press releases are not mentioned in WP:PSTS. Insofar as the definition of secondary sources on WP:PSTS goes: "Secondary sources are accounts at least one step removed from an event.[3] Secondary sources may draw on primary sources and other secondary sources to create a general overview; or to make analytic or synthetic claims.[4][5]", most press releases do not fit it. Please see the examples of typical press-releases I gave above. Please provide more evidence of your belief that press-releases are secondary sources. Paul Gene (talk) 11:00, 29 August 2008 (UTC)

Eubulides, your opinion that news articles constitute secondary sources goes against the prevailing expert consensus. According to this definition,[7] "In science, secondary sources are those which simplify the process of finding and evaluating the primary literature. They tend to be works, which repackage, reorganise, reinterpret, summarise, index or otherwise “add value” to the new information reported in the primary literature." Most of the articles in the popular press tend to simply report the news, with the information usually taken directly from a press release or an abstract. In addition, they are contemporaneous with the object/phenomenon, which is one of the distinctive characteristics of a primary source.("A secondary source is a work that interprets or analyzes an event or phenomenon well after the fact. It is generally at least one step removed from the event." [8]) This source [9] directly lists mass media as a primary source: Examples of primary sources: ... Mass media ... The products of the mass media can be primary source documents if they were produced at the time of the events or phenomena in question. Examples are: newspaper and magazine articles". Another source [10] also states Some examples of primary sources include:...Contemporary magazine and newspaper articles." I was not able to find any support for your argument that, unlike in historical sciences, in medicine the news articles are secondary and not primary sources. Paul Gene (talk) 02:45, 29 August 2008 (UTC)

Wiki's definitions are at WP:PSTS; we are in line with them, which is where we should be. SandyGeorgia (Talk) 02:51, 29 August 2008 (UTC)

In other fields, e.g., political reporting, newspapers are indeed primary sources. For example, to report Barack Obama's position on the educational vouchers, Political positions of Barack Obama cites a Chicago Tribune article about a 2004 debate; this was a primary source on that debate. In contrast, in medicine, typically refereed journal articles are the primary sources, and press releases and/or newspapers report about the journal articles, adding commentary about their significance; these are secondary sources. For example, Morrow et al. 2008 (PMID 18621663) published in Science a study about homozygosity mapping to identify shared genes in autism, and that same day a story about the study appeared in The Times. The Times story was a secondary source: it commented about the study, with quotes like "This publication a big event in the world of autism research", without directly accessing the original data that the study reported. There may be occasions where a newspaper would be a primary source on a medical topic, but these would be rare, I'd think; not worth mentioning or emphasizing here. Eubulides (talk) 03:27, 29 August 2008 (UTC)

Sandy and Eubilides, you are not in line with the definition of secondary sources at WP:PSTS: "Secondary sources are accounts at least one step removed from an event.[3] Secondary sources may draw on primary sources and other secondary sources to create a general overview; or to make analytic or synthetic claims.[4][5]" As I already mentioned, most of the stories in pop press are news articles, they are contemporary and thus are not sufficiently removed from the event to count as secondary sources. They generally do not provide analysis based on several other primary sources or create an overview of the topic. Many news articles in the political area often make comments similar to the ones made in the science Times story you are referring to. For example, in another news story http://www.timesonline.co.uk/tol/news/politics/article4615384.ece] Times comments: " The commission’s push for a national register, however, could be more controversial.", without directly accessing the original commission's report. Political reporting in the pop press in that respect is not any different from scientific reporting. Nevertheless, as the thrust of the press reports is on the news they are firmly considered to be primary sources. In order to re-define the pop-press reports of scientific news as secondary sources you have to provide the evidence that they are sufficiently different from other reporting. You have to back that up with the quotations showing that scientific reports in pop-press are considered by experts to be secondary sources, otherwise you are engaging into original research. In what I was able to find so far on the expert definitions of secondary sources as applied to science, the pop-press reports appear to be firmly ruled out (see the definitions in my previous post). Paul Gene (talk) 11:00, 29 August 2008 (UTC)

Whether we regard press releases as primary or secondary isn't the most important thing we can say about them. The vital point is that they are a biased source of information written at lay level which may not contain enough contextual information or balance of viewpoints in order to write a good encyclopaedic article. If the press office for hospitals/research labs are writing about newly published work, they are often effectively writing the journalist's article for them, so would be considered secondary sources. If they are reporting on unpublished data, then they are more like primary sources. If they contain quotations from the researchers, as though they had been interviewed about it, then the are also more like primary sources for that detail. Rather than trying to divide all published material cleanly into primary and secondary, it may be better to avoid the issue with certain sources, and concentrate on their strengths and weaknesses without classification. Colin°^Talk 14:04, 29 August 2008 (UTC)

I agree with Colin's points. I looked at WP:MEDRS and its text currently does what it says: it does not say that the popular press or press releases are primary sources, or secondary sources. Perhaps some minor change to WP:MEDRS would help clarify this issue (to avoid further questions along these lines), but offhand I don't see what change that would be. Eubulides (talk) 18:26, 29 August 2008 (UTC)

I agree with Sandy: WP:MEDRS's version of PRIMARY just needs to line up with the normal WP:PRIMARY, while providing some useful information about how it might apply to a medicine-related article.

Press releases are difficult to assess in the PSTS model. A press release from the FDA about fining the American Red Cross Blood Services for long-standing cGMP problems is probably a primary source. If the American Red Cross Blood Services issues a press release whining about it, then their press release is a primary source for their response, but probably a secondary source for the fact of the fine. If a third organization issues a press release to chide the FDA for dragging their feet, then the third press release is a primary source for their opinions, but definitely a secondary source for the fact of the fine.

None of that changes the fact that none of these are issued by independent third-party sources with reputations for fact-checking (something that is mentioned in WP:V because it's a relevant legal standard). WhatamIdoing (talk) 19:46, 29 August 2008 (UTC)

• Sandy, what is your problem with deleting the questionable paragraph and just leaving it up to WP:PRIMARY to deal with? I thought that was essentially what you suggested. WhatamIdoing also agrees that press-releases are difficult to assess in the PSTS model. Colin also says that "Whether we regard press releases as primary or secondary isn't the most important thing we can say about them.". Eubulides says "I looked at WP:MEDRS and its text currently does what it says: it does not say that the popular press or press releases are primary sources, or secondary sources." Remember, we should not engage into original research and re-write conventional definitions of secondary and primary sources. No examples of expert support of the questionable definitions has been given in this discussion in spite my repeated requests and questions.Paul Gene (talk) 00:53, 30 August 2008 (UTC)

The text you deleted has so far, been questioned only by you. It is stable text, and has been in the article in its current form for over a year. It is not a definition; it is advice from experienced Wikipedia editors. It does not say that press releases or the popular press are secondary sources; it says that the secondary source material in press releases and the popular press tends to be less useful than the other secondary sources mentioned (e.g., refereed articles in medical journals). This is widespread consensus among experienced editors of medical articles. Certainly I find it to be true: for example, this article in the Washington Post is far less useful than this article in the Annual Review of Public Health; they are both dated 2007 and they both talk about the epidemiology of autism, but the medical-journal review article is way, waay, waaay more informative and reliable. Eubulides (talk) 01:58, 30 August 2008 (UTC)

Eubulides,

(1) It is not true that Paul Gene is the only one who questions (actually, objects) to that text. I also object to it.

(2) I've known Ed Edelson for many years, since he was science reporter for the New York Daily News, and I'm familiar with his work. I think this Washington Post story on the rise in autism is a good, useful story. It's a good report of what experts said at a meeting, and it gets a range of authoritative views. You have not expressed any objection to it on the merits. You have merely used it to support your personal opinion that you don't like newspaper medical stories.

You don't define "useful." The Washington Post story is free on line. The Annual Reviews article is not. Unless Wikipedia editors have a subscription to Annual Reviews, they can't access it. That violates WP:VERIFY. A source that is inaccessable and cannot be verified is not useful.

You have provided no evidence that newspaper articles are less "useful" than academic reviews. In fact, you've provided evidence against it. I propose, based on your own comparison of Washington Post to Annual Reviews, that newspapers do provide useful material, and therefore we should delete the incorrect text as Paul Gene did.Nbauman (talk) 18:10, 30 August 2008 (UTC)

Your claim that offline and non-free sources fail WP:V is nonsense. There is not one word in that policy that says we cannot cite journals that would require either a paid subscription or a trip to the library. WhatamIdoing (talk) 00:23, 31 August 2008 (UTC)

The question is whether newspapers are "useful". Eubulides doesn't define the term "useful", and he has not cited anything in the Washington Post story that is wrong, nor given any reason why it or similar articles should not be used in a WP article. I think it's clearly useful, in the ordinary sense of the term. I would like to see Eubulides's specific criticism of the article. So far, he hasn't given us one.

I agree that peer-review subscription-only articles should be used, and are usually more reliable, although that has the problems of WP:LINKS Sites requiring registration. The question is whether newspapers should also be used.

I would like to see a good, logical argument for why newspapers should not be used. So far, nobody has given us one. Nbauman (talk)

While we're wishing, I'd like to know how many times WP:EL has to state The subject of this guideline is external links that are not citations of article sources and This guideline does not apply to inline citations, which appear in the "References" or "Notes" section and A site that requires registration or a subscription should not be linked unless...[it] is being used as an inline reference (and so forth) before editors will realize that it does not apply to references. May I encourage you to read both paragraphs in the section you linked? WhatamIdoing (talk) 20:56, 31 August 2008 (UTC)

Newspapers

Journalists, writing in the popular press, and corporate press releases tend to provide less useful secondary source material. Such material may be appropriate for inclusion in some contexts.

As a journalist, I disagree with that statement, and I agree with Paul gene that "This defintion is questionable and OR. You have to present evidence in its favor before including it.

I'd like to know why somebody made deletions in the following paragraph. I didn't see anything about this in the history or my watch list, so I didn't realize it had been deleted.[11]. This is a WP:RS authoritiative academic review of the quality of newspapers, as distinct from an editor's personal opinion.

Newspapers

The quality of newspaper coverage of medicine ranges from excellent to irresponsible, and they should be verified like any other sources. Even peer-reviewed journals like the New England Journal of Medicine cite articles in newspapers like the New York Times and Wall Street Journal. Some tabloids, like the New York Daily News, have a reputation for careful fact-checking and knowledgeable reporters; others do not. Newspapers should be judged on the facts, not on prejudices. Some academic organizations that evaluate news coverage are Health News Review (U.S.), Media Doctor (Australia), and Media Doctor (Canada). The British Medical Journal reviews U.K. media coverage.

Nbauman (talk) 01:24, 30 August 2008 (UTC)

The material in question was deleted in this edit with the comment "keep it simple", a sentiment I tend to agree with. This should be a guideline, not an academic paper studded with citations. Besides, the longer text is self-contradictory: "Newspapers should be judged on the facts, not on prejudices." collides with "Some tabloids, like the New York Daily News, have a reputation...." Let's just keep it simple and say "quality newspapers". Eubulides (talk) 01:58, 30 August 2008 (UTC)

We've been round this debate already. See the archive. Newspapers simply aren't designed to be source material for medical or science articles in an encyclopaedia. I really struggle to understand how you think they are. And for anyone who thinks the UK press should be used as a source for medical facts I suggest you read today's article by Ben Goldacre in The Guardian. Colin°^Talk 12:56, 30 August 2008 (UTC)

Wikipedia should be simple, but we must follow WP rules of verifiability, WP:RS and WP:NPOV.

The statements about newspapers in the present articles are WP:OR, personal opinion, unsourced and not verifiable. The deleted text is an academic and professional consensus, supported by verifiable WP:RS and academic research and authority.

We did have a debate about this before. Your argument was to cite selected examples of newspaper stories that, in your opinion, were wrong in some way. My argument was to cite international academic studies such as Health News Review, which you deleted.

In Wikipedia, the personal opinion of one or a few editors can't prevail against verifiable WP:RS and WP:NPOV.

Whether you like it or not, Wikipedia editors use newspaper articles extensively to write and source WP articles. If you read the daily newspapers, and also the medical journals, as I do, you can see that the medical facts in newspaper stories are often more accurate than the facts in some journal articles, as the medical journals themselves often admit in retrospect (for example the NEJM articles on VIOXX). So if you accept major peer-review articles as reliable sources, you must accept their conclusions that newspapers can be reliable.

Since Wikipedia editors do use newspapers, this guideline should give them specific guidance about how to evaluate newspaper stories -- a vague statement about "quality newspapers" is useless, since it merely transfers the argument to whether a given newspaper is "quality." Why not tell them how to use WP:RS to decide for themselves whether a newspaper is quality?

Our responsibility here is to elaborate on WP:RS#News_organizations for medicine. Medicine is a specialty, with specialized rules, and particularly complicated information sources. We should give them useful, helpful guidance, from available academic sources, to decide for themselves, not just tell them to follow the authority of "our" peer-reviewed literature.

Wikipedians use newspapers as sources for WP articles because most people don't have access to the academic, subscription-only sources that you presumably do. This guideline could be harmful if pedantic editors use this as an excuse to delete good newspaper stories as sources, and replace them with subscription-only sources that ordinary readers can't verify, which would violate WP:VERIFY.

To address Colin, I would like to know how you support the claim that newspapers "aren't designed to be source material for medical or science articles in an encyclopedia." I work for newspapers, and we are aware that we have many readers who use our stories for many purposes. I also know people who worked for encyclopedias and reference books. They would certainly accept a Wall Street Journal or New York Times story for fact-checking purposes.

As for Ben Goldacre's story in the Guardian, first I would point out that you're contradicting yourself. You say that newspapers are not a source for medical facts. But you are using Goldacre's story yourself as a source for medical facts.

Second, if Goldacre is correct, the debunked link between MMR and autism was published in peer-reviewed journals, including review articles. So some newspaper stories (including Goldacre's) were more reliable than peer-reviewed journals.

Third, Goldacre says that the anti-MMR lobby targeted general journalists, rather than health or science correspondents (such as himself). This supports the conclusions of Health News Review, which you deleted.

But the fundamental problem is that you're simply citing a single example (published in a newspaper at that). You need more than a data point of 1 to prove your point. If you really believe what you're saying, then go to the peer-reviewed academic communications literature, as I did, and see what they say.

In short, the statements about newspapers in this entry are personal opinion, and wrong. You are deleting authoritiative, WP:RS, useful information that will help editors evaluate the reliability of newspapers for themselves. Nbauman (talk) 15:20, 30 August 2008 (UTC)

Contrary to the apparent beliefs of two editors, NPOV and NOR apply to articles, not guidelines created for internal use. Guidelines are not required to cite their sources, to find independent, third-party publications to support their claims, or to fairly present anything at all. They are, indeed, permitted to stand strongly against deprecated practices, to oppose things deemed undesirable, to announce good practices, and to base themselves entirely on original research. (Just the other week, WP:LAYOUT was counting up the relative proportion of articles using different headings for references in an effort to make a decision, which is clearly "original research".)

When a newspaper story reports a publication, then a responsible science editor will double-check the newspaper reporter's understanding by looking up the actual publication instead of blithely assuming that every single reporter always gets every single scientific fact correct the first time. (Nbauman, do you assert that you have never published a single factual error?) WhatamIdoing (talk) 00:35, 31 August 2008 (UTC)

• Somehow this discussion has overspilled from the previous topic. Thank you, WhatamIdoing for raising a good argument. This is the kind of discussion I would like to have in this space. Paul Gene (talk) 11:49, 31 August 2008 (UTC)
• And I concede you this point. It is then my original research (which I of course believe is better ;) ) vs. Eubulides's. It is then my and Nbauman's opinion vs. yours, Eubulides's, Sandy's and other editors. Then, we have reasonable people like Colin, who would not mind not having newspapers and press releases mentioned in the definition of secondary sources. It is also a poor style to include something which is often a primary source (as I believe everybody agrees) into the definition of the secondary source. I would suggest as a compromise option either deleting or moving this sentence from the definition part to Avoid citing popular press, which it anyways duplicates. Paul Gene (talk) 11:49, 31 August 2008 (UTC)

I just realized from the other discussion that this is a guideline, and should not be changed without discussion and consensus in Talk. The text about Health News Reviews was added by consensus and has been in the guideline for over a year without objection.

The burden of proof is not upon me to gain a consensus that the text about Health News Reviews should be restored; the burden of proof is upon Colin to gain a consensus that it should be removed.

I am restoring the text. If anyone wants to delete it, please discuss your reasons here and get consensus. Nbauman (talk)

I have made an edit regarding press releases that should keep Paul happy. I've moved mention of it down to the Popular press section.

As for Nbauman's comment: This isn't a guideline (yet) and there is no rule on WP that all edits to guidelines require discussion and consensus in Talk. Nbauman's text was originally added against consensus (as can be see on the archive) and was much discussed already. Life is too short to repeat all that, I'm afraid. Nbauman continues to misunderstand the word "reliable" and continues to cite the current-affair/investigative journalism example of Vioxx. Nbauman provided the example of a good pop press article, not me (Bell Curve). It was then promptly destroyed as a good source for WP. Goldacre was cited not for his medical facts, but his article on newspapers. Is it so hard to understand that newspapers are good for some things and not others? I don't think the English Literature wikiproject has this trouble, or the Mathematics wikiproject, etc, etc. Other WP:MED editors, feel free to remove the offending text. Colin°^Talk 14:24, 31 August 2008 (UTC)

Nbauman, I would be most happy if WP cited newspapers for their medical facts with precisely the same frequency as "the New England Journal of Medicine cite[s] articles in newspapers like the New York Times and Wall Street Journal.". That would improve the quality of our medical articles immensely. Colin°^Talk 15:23, 31 August 2008 (UTC)

No consensus for text about newspapers

I went back and read the records, and did not find support for Nbauman's claim "The text about Health News Reviews was added by consensus". Nbauman's original edit, which added this material in April 2007, was clearly controversial at the time. Newspapers, the first thread about it, had commentary by three editors, only one of whom (Nbauman) favored the change. Avoid citing the popular press had commentary by the same three editors, with the same results. Not ready for guidelines (May 2007), the only thread that explicitly mentions Health News Review, has commentary by four editors; again, only one (Nbauman) favored inclusion of such material. What appears to have happened is that a single editor tirelessly advocated the change and installed it without consensus.

Given all this history, along with the discussion above, I'm inclined to revert the controversial edit. The text in that edit sticks out like a sore thumb: it's argumentative, it gives a mistaken impression to unwary readers that the New York Daily News is in the same ballpark as the New England Journal of Medicine for reliability, and basically it encourages Wikipedia editors to substitute newspaper articles for medical-journal articles if they think the newspaper is reliable. Eubulides (talk) 19:17, 31 August 2008 (UTC)

Proposed replacement for Newspapers

I am not opposed to mentioning Health News Review, but it needs to be placed in context better. There are specialized journals such as Journal of Health Communication and Health Communication, but the best coverage of how media covers tends to appear all over the place, in journals ranging from the American Journal of Disaster Medicine (e.g., see Gionis et al. 2007, PMID 18488831) to the American Journal of Public Health (e.g., see Wenger et al. 2001, PMID 11211641). So, after reverting, I propose replacing the contents of WP:MEDRS #Newspapers with the following text.

The quality of newspaper coverage of medicine ranges from excellent to irresponsible. Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, medical facts and figures should be sourced to medical journal articles, perhaps with an additional citation to a free newspaper source for the benefit of readers who lack access to medical journals; one way to do this is with the "laysummary=" parameter of the Cite journal template. Sources for evaluating health-care media coverage include Health News Review, Health Communication, and the Journal of Health Communication; the best reviews of media coverage often appear in more-general sources such as the American Journal of Public Health and The Guardian.

Further comments are welcome, of course. Eubulides (talk) 19:17, 31 August 2008 (UTC)

(An updated version of this draft is in #Proposed replacement for Newspapers 2 below). Eubulides (talk) 08:46, 1 September 2008 (UTC)

First, clarify "more-general." I read or have read several of the major peer reviewed journals, and they all have occasional articles about the media, and while useful, they only cover one specific topic at a time, and do it irregularly. Health News Review and its sister journals, which are also peer-reviewed, cover the media regularly. In that sense they are more-general sources. I don't understand what you mean by saying that AJPH and The Guardian are "more-general."

Second, I don't mind mentioning The Guardian, which I read sometimes, but (1) do they have much analysis of medical news besides Goldacre? (2) The Guardian isn't peer-reviewed, like Health News Review, and to me that makes a big difference. Nbauman (talk) 20:37, 31 August 2008 (UTC)

• I changed "more-general" to "other" in #Proposed replacement for Newspapers 2 below.
• I mentioned The Guardian only because it came up in earlier discussion here. It's nice to mention a non-U.S. source, but if there's a preference to mention some other source that'd be fine too.
• I'd rather mention at least one non-peer-reviewed source in the list; some of the best work in this area (such as Goldacre's) is not peer-reviewed.

Eubulides (talk) 08:46, 1 September 2008 (UTC)

The statement "medical facts and figures should be sourced to medical journal articles" is too restrictive and not required given the reset of this proto-guideline. I think the lay summary suggestion is good, though the text needs a little work. Even readers who have access may wish a easy-to-digest version as WP is written for the general reader. In fact, this is one area where a press release might be useful (see discussion below).

I'm not sure what the "sources evaluating health-care media coverage" has to do with WP:MEDRS. By all means summarise their conclusions in this guideline. To answer Nbauman, Goldacre's remit in the Guardian is the discussion of bad science, usually wrt medicine. This often discusses the bad reporting of bad science in newspapers. His work is accessible and may be an eye-opener for some, but it is certainly an opinion-piece rather than a work of scholarly analysis. I'm not aware that any UK newspaper has a qualified full time science or medical reporter let alone anyone capable of editing such articles. Even good quality medical articles are mediocre sources for WP -- see the Bell Curve example in the archive. A good quality lay medical article can be well worth reading, in order to work out how to discuss the topic in lay terms. Colin°^Talk 20:55, 31 August 2008 (UTC)

• In #Proposed replacement for Newspapers 2 below, I changed "medical facts and figures should be sourced to medical journal articles" to "They should not be used as a sole source for medical facts and figures", which is nearly what is there now (with the addition of a "sole").
• Now that you mention it, the "sources evaluating health-care media coverage" stuff seems to be more relevant to WP:MEDRS #Assess the quality of evidence available than to WP:MEDRS #Newspapers. I revised the proposal to summarize their results in WP:MEDRS #Newspapers, and to move the "sources evaluating health-care medica coverage]] stuff to the other section.

Eubulides (talk) 08:46, 1 September 2008 (UTC)

In general I like this approach. However, I'd change However, medical facts and figures should be sourced to medical journal articles, perhaps with an additional citation to a free newspaper source to be less specific.

Medical facts can be properly sourced not just to journal articles, but also to college-level (and higher) textbooks and other reputable reference works (i.e., Gray's Anatomy). This expansion is particularly important for facts involving basic science.

Similarly, newspapers are not the only possible source of open-access information: there are many highly reputable health-related websites that might be at least as appropriate (*.cdc.gov, for example). Additionally, I'm happy to see good, open-access media articles listed in the ==External links==, which is another way to put lay-language information into the hands of interested readers. WhatamIdoing (talk) 21:16, 31 August 2008 (UTC)

#Proposed replacement for Newspapers 2 below changes the "medical facts and figures" text along the lines you suggest. I prefer to reserve External links for websites and the like, and to cite open-access media articles as usual with citations, except with a URL so that the reader can easily see that the media articles are freely readable (but surely this is a style issue?). Eubulides (talk) 08:46, 1 September 2008 (UTC)

Proposed replacement for Newspapers 2 etc.

With all the above in mind, here is a revised proposal for replacement for WP:MEDRS #Newspapers:

The quality of newspaper coverage of medicine ranges from excellent to irresponsible. Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and should not be used as a sole source for medical facts and figures. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, for example with the "laysummary=" parameter of the Cite journal template.

And here is proposed additional text to be appended to the last paragraph of WP:MEDRS #Assess the quality of evidence available:

Sources for evaluating health-care media coverage include Health News Review, Health Communication, and the Journal of Health Communication; the best reviews of media coverage often appear in other sources such as the American Journal of Public Health, Columbia Journalism Review, and The Guardian.

(Please see a proposed further revision below.) Eubulides (talk) 21:02, 1 September 2008 (UTC)

(This latter text wikilinks to a Health News Review article I just created.) Again, comments are welcome.

Eubulides (talk) 08:46, 1 September 2008 (UTC)

I think this addresses all the issues. Great writing! Paul Gene (talk) 09:17, 1 September 2008 (UTC)

(1) Along with Health News Review (U.S.), I included Media Doctor (Australia), and Media Doctor (Canada), to make it less U.S.-centric.

(2) I don't understand why you cite Health News Reviews, and then say that "the best reviews of media coverage" often appear in other sources. That says that HNR is not publishing "the best," while the others are. HNR's only job is to regularly evaluate health news, by a structured peer review process that includes medical doctors and journalists. I think that evaluation which is peer-reviewed is better than evaluation that is not peer reviewed. Do you disagree? If anything, HNR is "the best"; I wouldn't single it out but I would like to know why you single out the others as "the best."

As far as I know, the only reporting on health coverage in Columbia Journalism Review is by Trudy Lieberman (which is very good, but not peer reviewed). Can you cite anything else? The only reporting on health coverage in The Guardian is by Goldacre (which is not peer reviewed). Can you cite anything else? Nbauman (talk) 16:02, 1 September 2008 (UTC)

• We don't have room to list all such sources. One example of each major type should be enough. Come to think of it, this should be worded more clearly; please see below.
• The definition of "the best" depends on what kind of review one is looking for. Goldacre's column, cited above, is very high in quality, but as far as I know broad-scope coverage like that is not published in Health News Review.
• Although Health News Review has a team that produces high-quality reviews, these reviews are not themselves peer-reviewed, as far as I can see. This is in contrast to peer-reviewed publications such as the Journal of Health Communication and the American Journal of Public Health. That being said, I agree that "the best" doesn't need to appear here.
• With the above in mind, here's an updated version of the "Sources for" sentence.

Sources for evaluating health-care media coverage include review websites such as Health News Review and specialized academic journals such as the Journal of Health Communication; high-quality reviews can also appear in sources as various as the American Journal of Public Health, the Columbia Journalism Review, and The Guardian.

Eubulides (talk) 21:02, 1 September 2008 (UTC)

If space is a consideration and we don't have room for the Australian and Canadian reviews, then we should shorten it to:

Sources for evaluating health-care media coverage include review websites such as Health News Review and specialized academic journals such as the Journal of Health Communication; reviews can also appear in the American Journal of Public Health, the Columbia Journalism Review, The Guardian, and others.

Nbauman (talk) 04:22, 2 September 2008 (UTC)

Yes, that'd be fine. Eubulides (talk) 05:16, 2 September 2008 (UTC)

With respect to the following:

Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and should not be used as a sole source for medical facts and figures.

I'd like to know the evidence you have to support the statement that "most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits."

I think we all agree that 'some news articles do discuss exactly those issues, and some do not. For example, Health News Reviews evaluated the New York Times article on HPV vaccines, and it covered all those issues and more. [12] Health News Reviews found many more articles that met those issues.

On the evidence of Health News Reviews and other sources, we can say that some news articles fail to discuss those things, but I want to know the evidence for saying most.

As I understand it, this is based on the personal opinion of several editors. Is it based on anything else? Nbauman (talk) 17:02, 2 September 2008 (UTC)

That was not based on my (or other Wikipedia editors') opinion. It was based on Schwitzer 2008 (PMID 18507496), a source cited in the Health News Review article. That source's summary says "journalists usually fail to discuss costs, the quality of the evidence, the existence of alternative options, and the absolute magnitude of potential benefits and harms." Eubulides (talk) 17:31, 2 September 2008 (UTC)

Additionally, that statement appears to line up with everyone's experience (except perhaps for our professional journalist's) and this guideline is not an encyclopedia article. We do not actually have to provide reliable sources to support statements that we all agree on. The newspaper-seller's apparent horror of absolute risks drives me nuts. "Risk tripled!" the headline trumpets -- but never mentions that it's from 1 in 30,000 to 1 in 10,000. WhatamIdoing (talk) 17:58, 2 September 2008 (UTC)

Very good. That's a good source, open source, and an important point. I'm glad I pressed you on it. I think we should include the source. It doesn't lengthen the body of the text significantly to insert a footnote, and it increases the reliability and authority infinitely.

WhatamIdoing, could you or somebody else help me out? Could you link to the WP rule that says that factual statements in guidelines are exceptions to WP:RS? Nbauman (talk) 19:10, 2 September 2008 (UTC)

Content policies and guidelines apply to article space (i.e. the encyclopaedia). Read any one of them and you'll see the word "articles" repeated again and again. The policy and guideline pages, along with talk pages, user and project pages are all part of the machinery but not the product. The overriding rule for policy and guidelines is that the text is achieved through consensus. Colin°^Talk 19:40, 2 September 2008 (UTC)

Thank you, Colin. Can you link to an official statement in Wikipedia that says that? Nbauman (talk) 22:59, 2 September 2008 (UTC)

I can: First sentence, second paragraph of WP:RS: "Wikipedia articles should use reliable, third-party, published sources." Or let us have the third sentence of the second paragraph at WP:V: "Jointly, these policies determine the type and quality of material that is acceptable in Wikipedia articles."

So it appears that RS and V apply solely to articles. If you want to be WP:POINTy about it, let me suggest that you take it up at Wikipedia:Village pump (policy). I suggest that you start by telling them that you think that the provisions of WP:V itself need to meet WP:V. WhatamIdoing (talk) 00:19, 3 September 2008 (UTC)

WhatamIdoing and Colin, I believe that when WP:RS refers to "articles", that includes guidelines and policy statements insofar as the guidelines and policy statements refer to facts.

You believe WP:RS excludes guidelines and policy statements.

However, neither you nor anybody else can cite with a link any WP guideline or policy statement that explicitly supports your belief.

Is that correct? I just wanted to get that on the record. Thanks. Nbauman (talk) 09:10, 3 September 2008 (UTC)

• "Policy change comes from three sources: 1.Documenting actual good practices and seeking consensus that the documentation truly reflects them." Wikipedia:Policies and guidelines#Sources of Wikipedia policy Paul Gene (talk) 09:56, 3 September 2008 (UTC)

Nbauman, WP's content guidelines and policies explicitly apply to articles and don't need an explicit exclusion for the other parts of WP (which aren't articles) in much the same way that the Highway Code doesn't contain an exclusion for aircraft or boats. Colin°^Talk 12:20, 3 September 2008 (UTC)

Colin, my question was, is your contention that facts in guidelines do not require WP:RS (1) based on your own personal determination of what is common sense, or is it (2) based on an explicit WP rule that you can cite with a link. Your answer is therefore (1). Nbauman (talk) 16:59, 3 September 2008 (UTC)

For the record, I draw your attention to Wikipedia:Verifiability WP:V: "This page in a nutshell: Material challenged or likely to be challenged, and all quotations, must be attributed to a reliable, published source."

It doesn't distinguish between articles, guidelines, or anything else on WP, it just says "material."

So I think WP:V clearly states that any material challenged must be attributed to a reliable, published source -- including factual material incorporated into guidelines. Nbauman (talk) 18:11, 4 September 2008 (UTC)

This interpretation of WP:V is contradicted by material that is present in WP:V itself. For example, WP:V says "In general, the most reliable sources are peer-reviewed journals and books published in university presses". This is a factual claim, that is likely to be challenged using exactly the sort of challenges that we're seeing for WP:MEDRS. But WP:V does not cite a source for this claim. And the claim is not an isolated example: there are several such claims in WP:V, and none of them cite a source. Similiarly, WP:RS (which is a generalization of WP:MEDRS) cites zero sources for its several claims. Eubulides (talk) 19:30, 4 September 2008 (UTC)

WP:V's lead text says "Wikipedia:Verifiability is one of Wikipedia's core content policies. The others are Wikipedia:No original research and Wikipedia:Neutral point of view. Jointly, these policies determine the type and quality of material that is acceptable in Wikipedia articles." Note that last word. This is beyond tiresome. Colin°^Talk 20:01, 4 September 2008 (UTC)

I'm not convinced. I think the statement, that peer-reviewed journals and university presses are reliable, is so generally accepted that it does not require a WP:RS. However, the statement that newspapers usually have those flaws is not generally accepted, so it requires a source.

I don't want to argue about it forever. However, I do want to insert the source into that statement about newspapers. I think that's a reasonable compromise. One more footnote won't detract from the article. Nbauman (talk) 02:32, 6 September 2008 (UTC)

OK, there seems to be some consensus for the revised proposed text, as far as it goes, so I installed it. One minor wording change as I installed it: I changed "and should not be used" to "and news articles should not be used", to prevent a plausible misreading of the phrase to mean "and most news articles should not be used", a meaning which differed from the previous meaning and was not the intent. Eubulides (talk) 07:32, 3 September 2008 (UTC)

I have to throw in a few pennies, here. I think that anywhere on Wikipedia a statement is presented as fact, the factuality of that statement should be open to question, and if necessary those questions ought to be met with proper sourcing. even in policy and guidelines... in this case (it seems to me) Nbauman is objecting to what seems like a prejudicial statement against newspapers being presented as fact, and I'm not convinced he's wrong - wp:RS#News_organizations would seem to be on his side. I'm not sure that I would personally disagree with you that some newspaper articles on medicine are poor quality, mind you, but I can't really see how we can make that judgement as editors - on a case by case basis, or in total - without violating wp:SYN. I think this passage ought to be reverted while we discuss the matter a bit more. --Ludwigs2 00:16, 6 September 2008 (UTC)

I think we can reach consensus by adding the citation to support this claim: "However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits.[13]" I don't think we have consensus otherwise. Without the citation, it sounds like some editor's personal opinion and cranky anti-media bias. Nbauman (talk) 02:24, 6 September 2008 (UTC)

If you want to add a citation to back up what has already been agreed by the consensus of WP editors, then that is fine. I have no problem with that. The point we are making is that such citations are not required and WP could, if it wants to, collectively disagree with outside opinions (e.g., that lay people can make a useful contribution to a scholarly encyclopaedia :-) Colin°^Talk 10:32, 6 September 2008 (UTC)

I'd still be happier

I'd till be happier if this section were not so anti-newspaper. a newspaper is a reliable source, and it should always be assumed that a newspaper article has done appropriate fact checking to present a credible viewpoint. the objection that they don't present certain scientific facts is really more a concern for people doing primary research; it should not matter to a tertiariy source like wikipedia, and excluding newspaper articles on that basis is no justified. let me see if I can make a corrective edit. --Ludwigs2 21:47, 7 September 2008 (UTC)

Unfortunately the change you installed (without previous discussion) had serious problems. Here's what the change was:

However, most many news articles fail to discuss important scientific issues such as evidence quality, costs, and risks versus benefits, and a news article should not be used as a sole source for a medical fact or figure. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, and should generally be supplemented with academic secondary sources, for example with the "laysummary=" parameter of the Cite journal template.

Some problems with this:

• It changes "most" to "many". But "most" is accurate; see Schwitzer 2008 (PMID 18507496). It should not be watered down to "many".
• It removes mention of quality, costs, and risks versus benefits, but these are important and relevant issues, which Schwitzer mentions.
• It removes longstanding advice in this article that news articles should not be sources for medical facts and figures.
• It has things backwards, by suggesting that newspapers be the main source, with refereed journal articles being supplements. The more-reliable sources should be the main sources.

There is one change, the addition of the word scientific, which is not a serious problem, though I don't see why it's needed here. For now, because of the serious problems mentioned above, I reverted the change. Eubulides (talk) 23:16, 7 September 2008 (UTC)

it was a first draft, and I didn't expect it to be perfect. the main problem I have here is that you all seem to have unilaterally decided that [wp:RS#News_organizations]] does not apply to medicine articles. you seem to have decided this because you are (unfairly) equating newspaper articles in toto to poor quality newspaper articles. It might be advisable to do this on a case by case basis - possibly news articles require more careful vetting - but writing them off in advance as poor-quality sources is clearly against established policy. If you don't like my changes, that's cool, but how do we modify this passage to keep it in line with policy? --Ludwigs2 23:46, 7 September 2008 (UTC)

WP:RS is a guideline; it's not policy. The WP:MEDRS advice to not source medical facts and figures from newspapers has been in the page for a long time and seems to represent a working consensus here. It is appropriate to have a stricter guideline for biomedical topics. It's probably better to discuss drafts like that here first, before installing them; perhaps you could draft a proposed improvement with the above comments in mind? Eubulides (talk) 00:11, 8 September 2008 (UTC)

I've not seen a newspaper account of Tourette syndrome yet that was correct. Not one. No reason to prefer them when we have peer-reviewed sources. After viewing Ludwigs2's editing to WP:NPOV, I'm concerned about the effort to diminish NPOV and high quality sources. SandyGeorgia (Talk) 00:13, 8 September 2008 (UTC)

don't make this personal, Sandy. if you want to debate the value of particular edits, that's fine, but if you want to criticize my understanding or intent then you're way off-base (because - frankly - you really don't have the faintest idea what's going through my mind; if you did, you wouldn't be so quick to cast aspersions). let's try to keep things on track, ok? --Ludwigs2 20:28, 8 September 2008 (UTC)

WP:RS used to explicitly warn against using the popular press as a source for science. The remnants of that advice can be found in Wikipedia:Reliable source examples, which is just an essay. Regardless, both WP:V and WP:RS place an emphasis on scholarly works as sources. A quality newspaper is a vital reliable source for "news" (current affairs) and serves as a historical record of such events. It should no more be considered a reliable source for medical facts, than my Radio Times is a reliable source on Jane Austen. Colin°^Talk 08:47, 8 September 2008 (UTC)

I don't know what RS used to say, and I'm not trying to rest this discussion on what RS currently says (because, yes, policies change over time). to my mind, what reliable sourcing means is that sources provide informations which we can trust to be factually correct depictions of common knowledge in a field. academic journals and other scholarly sources are generally factually accurate, but they often lean towards primary research (and thus are not reflections of 'common knowledge'). newspapers reflect common knowledge well, but are not as factually accurate (in part because they are not peer reviewed, and in part because they focus on different facts than scientists do). do you think that's a bad assessment? --Ludwigs2 20:41, 8 September 2008 (UTC)

Again, WP:RS is a guideline, not policy. In my experience, the "common knowledge" about medical facts and figures that newspapers express but academic sources do not would be more-accurately characterized as "folk theories", or "speculation". For example, popular coverage on the causes of autism focuses heavily on environmental theories, which are are popular among non-experts but are quite speculative; in contrast, popular coverage tends to deemphasize genetic theories, which have a strong consensus among experts (see Singh et al. 2007, PMID 17237806). Eubulides (talk) 21:20, 8 September 2008 (UTC)

Controversies

I haven't been following the discussion very closely, so correct me if I've overlooked something. I assume the RS we're talking about are mostly concerned with dealing with scientific facts, in which case we want to use the best sources available, such as peer reviewed research, etc. Well, medical subjects, which (unfortunately) includes alternative medicine and other forms of quackery and health fraud, involves much controversy and there are numerous commentaries from mainstream sources, but such commentaries don't usually appear in peer-reviewed research. Such commentaries will also deal with the medical politics, turf wars, etc., which are also matters not usually dealt with in peer-reviewed research. Wikipedia's NPOV policy requires that we tell the whole story, and thus these controversies will need to be mentioned in medical and scientific articles. We will need to use non-peer-reviewed commentaries. Have you been dealing with this issue at all here? -- Fyslee / talk 19:50, 31 August 2008 (UTC)

So far the proposals have not explicitly addressed that situation; instead, they give generic advice that ideally would apply to alternative-medicine topics just as well as to mainstream-medicine. In Chiropractic I've found quite a few sources about medical politics etc. in peer-reviewed articles. That may be a special case, but at any rate the current guidelines don't advise against using non-peer-reviewed commentaries in that case, they merely advise not to use the popular press for medical facts and figures. So, although WP:MEDRS doesn't offer much guidance in your situation, it doesn't suggest not using the popular press either. Is that good enough for now? Eubulides (talk) 08:46, 1 September 2008 (UTC)

Press releases

Since I've used and fact-checked press releases extensively, and occasionally written press releases, I know the mechanics of writing a press release, and I can contribute some facts to the discussion.

Press releases represent the institution that issues the press release, and they are as reliable as that institution.

Peer-reviewed journals like BMJ, Lancet, JAMA, Science and Nature issue press releases, which are approved by both the editors of the journal and the authors of the papers. I think we can agree that those PRs are as reliable as the journals.

Universities issue press releases, and when they are about scientific subjects, usually in peer-reviewed journals, they are reviewed by the investigators who did the work, so they are as reliable as the investigator and usually as reliable as the journal article itself.

Book publishers issue press releases. Since they are reviewed by the author, they are as reliable as the author. (So saying "Press releases are accurate" is like saying "Books are accurate.")

Major pharmaceutical companies issue press releases, which are reviewed by their legal departments; they have to meet many legal requirements including FDA labeling restrictions and securities laws requirements. Corporations would be vulnerable to legal actions and damages if they knowingly printed inaccurate information, so they are usually literally accurate, although they can also be promotional and are not as objective as peer-reviewed journals. They're works of advocacy, which is not the same as the whole truth. If the pharmaceutical company wants to tell the whole truth, which they sometimes do, they'll give the benefits and the limitations of a new drug.

Finally, anybody can issue a press release. Any nut with a 50kg drum of DHEA and a pill press can put out a press release touting the benefits of DHEA. So it's hard to generalize about press releases and they have to be taken on their merits.

I see press releases, and then see the stories that are written about them in the newspapers and elsewhere. First, the wire services take about 10% of the medical press releases and run them pretty much as is. (Science Daily seems to be entirely compiled from press releases.) Then, if a story is particularly important or controversial, they'll do some fact checking and get comments (as Edward Edelson checked the meeting presentations in that Washington Post story.) Major newspapers like the New York Times and Guardian treat press releases that same way.

Sometimes the news sources get it wrong and the original press release is more accurate. Sometimes the news sources add useful perspectives to the story. Many science reporters have PhDs and MDs (for example Robert Altman of the New York Times) so their stories can often be more reliable and accurate than the original peer-reviewed journal article.

The main function of a press release is to explain the scientific publication in language that a non-specialist reader can understand. (Wikipedia is also written for a non-specialist audience.) Scientific journals have certain conventions which make them hard to understand, such as using technical language and putting the conclusion at the end. Press releases explain technical terms (as WP should) and put the conclusion at the beginning. They usually know that laymen will frequently misinterpret certain things from journal articles, so press releases should (but don't always) articulate those points clearly, for example that a study has only been done in vitro.

Significantly, press releases are designed to be copied, so under the copyright laws "fair use" would be very broadly interpreted.

Furthermore, they often contain the significant content of a subscription-only journal article, in a form that is reliable and less subject to misinterpretation than the free abstract and sometimes the article itself.

For these reasons, I think that press releases from reliable sources are a good WP:RS for Wikipedia, in addition to the link to the PubMed or original article. Press releases are open access, and you can check them to make sure an editor didn't misunderstand or distort the original article.

That's why I also think that press releases that have gone through an additional layer of fact-checking, and published as news stories in a reliable newspaper, are also good WP:RS, in addition to peer-reviewed sources.

Are press releases about journal articles primary or secondary sources, for Wikipedia purposes? I don't know. The main problem with primary sources is that they are subject to misinterpretation, and using them is WP:OR. Press releases are less subject to misinterpretation, because they are designed to be read by non-specialists and designed to flag common misunderstandings. So they do have that advantage and serve that purpose of secondary sources. Nbauman (talk) 20:09, 31 August 2008 (UTC)

All this proto-guideline currently says is that press releases "can be a biased source". They are saying "look at what we've done" or "listen to what we have to say on X". As you say, some press releases are effectively self-published sources that any crank could create. Others have a degree of editorial (or, at least, legal) oversight. Regardless, they are about as far from independent and impartial as you can get. So we need to treat them with caution. Is there anything in this guideline wrt press releases that you disagree with. I've suggested above that they may be useful as lay-summary versions of scholarly or inaccessible sources. Do you want to suggest some extra text on PRs? If so, please make it brief as the above posting was verging on TLDR. Colin°^Talk 21:07, 31 August 2008 (UTC)

(BTW) Using primary sources isn't necessarily WP:OR. For more details, see WP:OR! Colin°^Talk 21:21, 31 August 2008 (UTC)

Again, the fact that a press release is an open-access source does not make it a reliable source. Those two qualities are entirely unrelated. WhatamIdoing (talk) 21:08, 31 August 2008 (UTC)

"Universities issue press releases, and when they are about scientific subjects, usually in peer-reviewed journals, they are reviewed by the investigators who did the work, so they are as reliable as the investigator and usually as reliable as the journal article itself. " They do not misrepresent the facts of the work, but they may well misrepresent the significance. They tend to tout the importance of the study without saying just what it is or exactly where it has been published. They are not intrinsically unreliable, but I';d never ever use one for a matter of opinion about the work. The claims in the journal article are peer-reviewed. The claims in the press release are whatever the author of the article pleases. DGG (talk) 08:12, 1 September 2008 (UTC)

My general experience is that a press release for a scientific study is not as good as the study it promotes. Here's an example: Autism says "A 2008 study found that compared to their peers, autistic boys have significantly thinner bones if on casein-free diets" and cites Hediger et al. 2008 (PMID 17879151). But the NIH press release for that study doesn't bother to say specifically whether that difference in thinness was statistically significant. To justify the word "significant" in Autism, one really must cite the research study, not the press release. And this is one of the better press releases I've seen; it was carefully and conservatively written and was obviously vetted by the researchers. Eubulides (talk) 08:46, 1 September 2008 (UTC)

I agree with Eubulides that a press release of a scientific study (from academia) often exaggerates its significance and skips on finer details. On the other hand, I agree with NBauman that corporate press releases are generally of a better quality as they avoid making any exaggerated claims for legal reasons and strictly stick to the facts for the same reasons. They often precede a peer reviewed article and may be the only information source of the new developments for quite some time. It is a bit of a paradox but corporate press releases are generally reliable and often useful sources. In regards to style, "Such articles may be based uncritically on a press release, which can be a biased source." sounds awkward, does not add any content, and can easily be deleted. I wonder if NBauman can come up with a short sentence on press releases in general to replace it. Paul Gene (talk) 09:45, 1 September 2008 (UTC)

Guideline name & location

Ok I've acted as an admin and promoted from a proposed guideline to an actual guideline (RfC closed and my thoughts above).

A quick question though, as now a guideline of wikipedia, should it remain in this a single project space or be promoted in its name & location, so loosing the subpage "/" ? Hence from Wikipedia:WikiProject Medicine/Reliable sources to (current word order might suggest Wikipedia:Medical reliable sources, but I think might be better by using the standard disambiguation convention of brackets) Wikipedia:Reliable sources (Medicine) ? David Ruben ^Talk 01:09, 1 September 2008 (UTC)

This is good news! Excellent work, everyone. I think we should stick with the standard disambiguation format. -- Fyslee / talk 01:58, 1 September 2008 (UTC)

MEDMOS is at Wikipedia:Manual of Style (medicine-related articles). Perhaps this should use a parallel name. WhatamIdoing (talk) 06:11, 1 September 2008 (UTC)

Agree with WhatamIdoing on David's second suggestion. Also, this page needs to link back to MEDMOS somewhere. Colin°^Talk 10:32, 1 September 2008 (UTC)

To keep to bracketed disambiguation style and in keeping with MEDMOS approach, I've moved this wikiproject subpage from Wikipedia talk:WikiProject Medicine/Reliable sources to Wikipedia:Reliable sources (medicine-related articles). We can (at this early stage) rename, but I think I prefer WhatamIdoing's suggestion to my previosu "Wikipedia:Reliable sources (Medicine)" which seems over-terse in structure. David Ruben ^Talk 23:57, 1 September 2008 (UTC)

Colin, is this what you meant for "link back to MEDMOS somewhere" ? David Ruben ^Talk 00:12, 2 September 2008 (UTC)

Just as Wikipedia talk:Reliable sources not general place to discuss specific sources (vs the overall policy), should this guideline talk page have a similar top tag-box to some form of /Noticeboard. We can either elect to point to Wikipedia:Reliable sources/Noticeboard or create (but then have to personally watch ourselves) Wikipedia:Reliable sources (medicine-related articles)/Noticeboard ? Feel noticeboard creep if we are not careful, so I think I would prefer the former existing RS/Noticeboard... David Ruben ^Talk 00:19, 2 September 2008 (UTC)

To be honest, I don't expect to get too many questions. The usual RSN is probably good enough, or people can leave a message at WT:MED (or WT:PHARMA, or other relevant project). WhatamIdoing (talk) 03:27, 2 September 2008 (UTC)

Thanks, with no further direct comments on this specific point, I've added[14] a top banner to this talk page, pointing discussion of specific examples (vs overall guideline discussion) to RS/N (as per WP:RS's own talk page) as well as options of WT:MED & WT:PHARM. Likewise in a similar style to the last line of WP:RS's leadin, to WP:MEDRS's leadin - rephrase at will. David Ruben ^Talk 22:33, 8 September 2008 (UTC)

---

I left a message on David's page asking for the precedents he followed to promote this useful but extremely raw essay over the objections of significant minority of editors (3 vs 14). Paul Gene (talk) 09:59, 5 September 2008 (UTC)

David gave a full explanation of his reasons further up at #Should we make this a guideline?. He specifically addresses your and Una's issue with secondary sources, and in common with most other editors finds WP:MEDRS to be in alignment with existing policies and guidelines in this regard. The other "oppose" is an outlier and can be ignored. Since both objects wrt primary vs secondary are not actionable without bringing this guideline into conflict with established policy, there seems little reason for them to block the adoption of this guideline. I'm sure this guideline can be improved and will be by collaboration and consensus. Lets work to improve it and move on. Colin°^Talk 12:16, 5 September 2008 (UTC)

Assess the quality of evidence available

I'd like to see WP:MEDRS#Assess the quality of evidence available expanded along the lines of #Use up-to-date evidence.

The primary point that I'd like to add is that if you don't have good quality evidence for a scientific or medical fact, then the Wikipedia article probably doesn't need to mention that fact. I'm sure you can all imagine a prototypical "poor evidence" scenario: an unblinded, nonrandomized, unverified, retrospective, unvalidated internet-based survey of a nonrepresentative sample of people that "proves" something.

This guideline (I think) would be a reasonable place to provide the basic advice that editors need: It's often best to omit low-quality evidence entirely, even if it means that the article is shorter or "incomplete".

What do you think? WhatamIdoing (talk) 05:42, 5 September 2008 (UTC)

That sounds like a rules creep to me. There are many cases when nonrandomized retrospective studies are the only sources, for example, for the rare drug effects. Another case is when such a study generates a considerable controversy and press coverage, and as such should be reflected. So, I do not see how this addition would be helpful to the editors. The overarching principles of editorial discretion and reaching consensus via discussion are sufficient to handle the situations described by What. Paul Gene (talk) 09:47, 5 September 2008 (UTC)

This assumes that most of medicine's decision making is founded on evidence-based medicine; it isn't. While it is useful for WP editors to have knowledge of what makes good-quality primary evidence, judging that primary evidence is really the job of someone writing a review (literature or systematic or whatever). Such a review may also make statements based on consensus opinion of experts or even from personal experience (esp. for the rare stuff that will never get a RCT). We can base our articles on such statements, even if the evidence behind them is lacking. There's also a place for opinion and supposition, if directly attributed. Colin°^Talk 10:11, 5 September 2008 (UTC)

I'd hate to have to write an article on surgery or emergency medicine or any of dozens of other medical fields if I were limited to citing only randomized controlled trials. See, for example, Vincent 2004 (PMID 15302748) "in intensive care medicine the situation is a little different, with RCT evidence frequently lacking" and Jeppsson & Thorlacius 2005 (PMID 18333189) "There are several reasons for this lack of RCTs in surgery." Eubulides (talk) 16:45, 5 September 2008 (UTC)

I agree with Paul Gene about rules creep.

Evidence-based medicine is not identical to randomized controlled trials. There are many reasons why RCTs are impossible, and doctors who follow EBM acknowledge that and use the best available evidence. For example, there is no RCT to demonstrate the harmful effects of cigarettes. And there is always the period from the time of the innovation to the time of the definitive RCT, when people have to make decisions based on the best evidence. I refer you again to that "Clinical decisions" article in NEJM about skin infections. Sometimes, when you diagnose the disease specifically enough, there is no good evidence.

I think our job should be to explain how strong the evidence is -- where it is along the path to definitive evidence. To do that might require a bit of background on EBM. And it might require some clinical detail such as whether something is supported by prospective or retrospective trials, etc. Nbauman (talk)

Reorganize newspaper-related text

As I understand it, the two sections WP:MEDRS #Use caution in citing the popular press and WP:MEDRS #Newspapers are intended to have two different functions. The former is supposed to provide basic advice about whether and how to cite the popular press, and the latter is supposed to describe the popular press's coverage of medicine-related articles. However, in reading the two sections, it appears that these two functions are not clearly separated, and material that should be in one section is put into the other (and vice versa), which is confusing. So I propose the following replacements for these two sections, with strikeouts indicating the old version and italics the new. These changes are intended to merely move text around, not to make any significant insertions or deletions.

---

Use caution in citing the popular press

The popular press is generally not a reliable source for science and medicine articles. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease, or an every-day substance as the "cause" of some dreaded disease. Newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source. They also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms. A news article should not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, for example with the "laysummary=" parameter of the Cite journal template.

On the other hand, high-quality popular press sources can be very useful in describing the sociocultural aspects of medicine, disease, and health. Common sense and the general guidelines presented in the verifiability policy and general reliable sources guideline should be considered in determining whether a popular press source is suitable for these purposes.

---

Newspapers

The quality of newspaper coverage of medicine ranges from excellent to irresponsible. Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits,. and a news article should not be used as a sole source for a medical fact or figure. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, for example with the "laysummary=" parameter of the Cite journal template. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease, or an every-day substance as the "cause" of some dreaded disease. Newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source. They also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms.

---

(end of proposal) Eubulides (talk) 17:30, 8 September 2008 (UTC)

I favour just merging the "popular press" section into one section later on (i.e., keeping it out of "Definitions and basics"). The "Periodicals" section could be split into two: "Scholarly journals" and "Popular press". The note about popular science magazines could be incorporated into the latter, along with the text on newspapers and other popular press issues. Colin°^Talk 18:06, 8 September 2008 (UTC)

OK, thanks, a revised draft is in #Popular press reorg 2 below. Eubulides (talk) 21:22, 8 September 2008 (UTC)

I think this still fails to recognize the distinction between press sources. The NYT for example is generally considered reliable for mecial coverage to the same extent as most science oriented news services. This is a little tricky though, as at present I am not aware of any other English language newspaper I would say the same about--most certainly I would no longer say this of the Guardian. DGG (talk) 18:14, 8 September 2008 (UTC)

The issue is not just reliability wrt medical facts, but also the suitability of the text as a scholarly source for an encyclopaedia. Can you think of a situation were, writing an FA-quality article, you would choose to cite the NYT rather than seek out a more authoritative source on medical information (excluding current affairs)? I assume you are referring to NYT articles rather than the NYT's online health guide, whose text is taken from A.D.A.M., Inc. Colin°^Talk 20:07, 8 September 2008 (UTC)

The NYT is quite good, but even it has problems at times. Its quality ranges from Elizabeth Rosenthal's fine piece on cancer vaccines last month (HNR review) to Andrew Pollack's relatively-mediocre piece on OraVerse in May (HNR review). And if you read the reviews you'll see that even's Rosenthal's piece missed an important aspect of the HPV vaccine issue; this is something that would have been caught in a good peer-reviewed journal. Eubulides (talk) 21:22, 8 September 2008 (UTC)

Yes I can think of such an occasion when the researchers have a strong conflict of interest. For example, a suicide of a healthy volunteer in the duloxetine trial (see Duloxetine#Suicidality) has only been described in a popular press. In this case, the conflict of interests prevented the normal flow of scientific information. If it did not happen in the Lilly labs, there would have been a case study in a scientific journal. As the things stand now, the popular press is the only source of such information. Paul Gene (talk) 11:20, 9 September 2008 (UTC)

Popular press reorg 2

OK, here's a revised proposal in the light of Colin's suggestions above:

1. Remove WP:MEDRS #Use caution in citing the popular press. Almost all its text will be moved to a new Popular press section as described below.
2. Replace WP:MEDRS #Popular science and WP:MEDRS #Newspapers with a new Popular press section shown below. This section's text is taken from the current WP:MEDRS #Use caution in citing the popular press section, with additions in italics and deletions struck out:

Popular press

The popular press is generally not a reliable source for science and medicine information in articles. Most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease, or an every-day substance as the "cause" of some dreaded disease. Newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source. They also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms.

A news article should not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example with the "laysummary=" parameter of the Cite journal template.

On the other hand, the high-quality popular press sources can be very useful in describing the sociocultural aspects of medicine, disease, and health can be a good source for social, biographical, current-affairs and historical information in a medical article. For example, popular science magazines such as New Scientist and Scientific American are not peer reviewed but sometimes feature articles that explain medical subjects in plain English. As the quality of press coverage of medicine ranges from excellent to irresponsible, cCcommon sense and the general guidelines presented in the verifiability policy and general reliable sources guideline should be considered in determining whether a popular press source is suitable for these purposes.

---

(end of proposal) Eubulides (talk) 17:30, 8 September 2008 (UTC)

• The popular press is generally not a reliable source for science and medicine articles doesn't seem quite right. Perhaps The popular press is generally not a reliable source for science and medicine information might be more accurate. An article that is primarily on a scientific or medical topic may include many facts that are not themselves scientific or medical in nature, such as the sales figures for a prescription drug. WhatamIdoing (talk) 05:23, 9 September 2008 (UTC)
  • Let's combine these to say "information in articles". After all, a Ladybird book can be a reliable source of information, but certain not suitable as a source on WP. Colin°^Talk 07:24, 9 September 2008 (UTC)

• OK, I added the "information in", in the draft above. Eubulides (talk) 16:49, 9 September 2008 (UTC)

I've been following this discussion for a while, with a mixture of reactions. I agree with WhatamIdoing that "information" would be better than "articles" in the above, but I suspect that the more central issue is that: The popular press is not >as< reliable as is the original research articles. In some cases, the original research article is more accurate (my own scientific findings are very frequently misinterpreted by the press), but the original research article is oftentimes too technical for most WP editors to interpret themselves.
— James Cantor (talk) (formerly, MarionTheLibrarian) 12:06, 9 September 2008 (UTC)

• My own experience is that the original research is almost invariably more accurate, and all too often a lot more accurate, than the news stories about it. If that's the case, we should be citing the original research, instead of relying on the news story. A news story can be helpful for establishing the nontechnical context, but for the facts and figures themselves it's almost never a better source.
• I agree that the original research article is often too technical for most WP editors. My experience, however, is that when that happens, the article is too technical for science reporters as well. One case in point is Abrahams & Geschwind 2008 (PMID 18414403), arguably the best current review of the genetics of autism. This is a source that I suspect no professional news reporter can read and understand; at least, if there is such a reporter, I don't know who it would be. In autism, science reporters tend to go off and write stories on hypothesized environmental influences, in part because that's stuff they do understand.

Eubulides (talk) 16:49, 9 September 2008 (UTC)

No further comment on the reorganization (though there continues to be objection to some of the text being reorganized; this appears to be an independent issue), so I installed it. Eubulides (talk) 22:43, 16 September 2008 (UTC)

Sorry I failed to comment; I just saw the change, beautiful work. SandyGeorgia (Talk) 23:07, 16 September 2008 (UTC)

What Schwitzer says

This proposed section misstates the source:

The popular press is generally not a reliable source for science and medicine articles. Most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits.

Schweitzer studied newspaper and TV news stories, not the popular press in general. His summary says nothing about popular science magazines, such as Scientific American, New Scientist, WebMD, etc.

I'm all for criticizing the press, but I also believe in objective research and having a solid scientific basis for your research.

I object to any statements about the popular press that simply reflects the personal opinions, prejudices or WP:OR of the editors.

There are lots of evaluations of the accuracy of popular scientific magazines. If you believe in peer review and the scientific method, you should read that literature and use that as the basis of your statements about the popular press.

For example, the standard reference book used by librarians to evaluate periodicals is Magazines for Libraries, which used to be edited by Bill Katz. The book has several chapters, at least one of which deals with scientific magazines, written by a librarian with a scientific degree, and I believe the chapter itself is peer-reviewed. (I have a copy filed away somewhere.) A statement about popular science books cited to Magazines for Libraries is reliable and meaningful. A statement based on the personal opinions of a WP editor (or even a majority of editors) is unreliable and meaningless.

Broad statements about the "popular press" or "popular scientific press" are also WP:WEASEL. How do you define "popular scientific press"? Is Scientific American or New Scientist the popular scientific press (and unreliable)? What about WebMD (which is reviewed by doctors)? What about IEEE Spectrum? Or the news sections of Science (which is often more reliable than the peer-reviewed articles they report on)?

Terms like "generally" are WP:WEASEL. They're using vagueness as a substitute for evidence. ("Let's compromise and say 'generally'.") Schwitzer doesn't say "generally," he says (based on research) that a majority didn't meet his predefined standards. Nbauman (talk) 15:36, 9 September 2008 (UTC)

• The citation supports a claim about "news articles", not "popular press".
• The proposed text clearly states that it includes Scientific American and New Scientist among the popular press.
• As Colin wrote, this is a guideline not an article, and it doesn't need to cite sources to support its advice. If this sort of citation-quibbling is going to be a problem, I propose that we simply remove the citation to Schwitzer 2008. The text doesn't need that citation to reflect the working consensus of Wikipedia editors in this area.

Eubulides (talk) 16:49, 9 September 2008 (UTC)

Consensus includes me. Consensus is not a majority vote. Nbauman (talk) 17:05, 9 September 2008 (UTC)

Yes, of course, just as consensus is not necessarily unanimity. All this should go without saying. However, as far as I can see, the proposed change doesn't significantly add, remove, or change the advice, other than by reorganizing its text. The arguments in this subsection seem to reflect a longstanding disagreement with the advice, and that disagreement should be independent of this proposed reorganization. Eubulides (talk) 18:33, 9 September 2008 (UTC)

Consensus means "general agreement". If your opinion is in absolute conflict with the "general agreement", then the consensus does not include you. Furthermore, repeated attempts to assert that opinion, in spite of arguments by many other editors, is verging on tendentious behaviour. It is all very admirable, standing up for your profession, but I just can't imagine the folk at Britannica are having this sort of discussion. This is an encyclopaedia. If it isn't written directly by scholars (and we have a few), then at least it can be sourced to the work of scholars. Colin°^Talk 20:03, 9 September 2008 (UTC)

Rfc Promotion of MEDRS to guideline

I would like to ask the community opinion regarding the promotion of this page to guideline status. This was done by DavidRuben over the objections of a significant minority of editors. Wikipedia_talk:Reliable_sources_(medicine-related_articles)#Should we make this a guideline?.

I content that this decision was unprecedented, extraordinary and contrary to the Wikipedia spirit and policies on consensus. I believe that the concerns of the dissenting editors were not adequately addressed. David's decision put a de facto stamp of approval on the majority's opinion, prevented us from reaching a mutually satisfactory accommodation and from further improving the quality of this page.

DavidRuben commented that: "admins weigh up proposals on the strength of the points raised, and frankly 2 of the opposing views were I thought weak and well argued against" "So just as AfD or AN/I (by admins) or RfA (by sysops) and other straw polls are decided finally by one person who weighs up the issues, but are not beholden to precise vote percentages, so I acted in this example." "Guidence can and should always be improved upon and that is what is currently happening in the talk page threads" User_talk:Davidruben#Promoting_MEDRS_to_guidelines_question

Paul Gene (talk) 11:39, 11 September 2008 (UTC)

• Oppose guideline status. This guideline violates the pillars of Wikipedia in many ways. Changes were made without consensus. Changes were made without proposing them in talk. DavidRuben asserts that the opposing views were "weak and well argued against", so he unilaterally made changes; that is the very opposite of consensus. We should revert to the last consensus version and start over. Nbauman (talk) 12:11, 11 September 2008 (UTC)

Not so; in fact, this page has been developed and discussed over several years, and enjoys broad consensus. Also, can anyone pont to any guideline page that has been subjected to such a process as this? They are guidelines; most Projects slap a tag on them and add them to MoS with far less discussion and consensus than this page enjoyed. SandyGeorgia (Talk) 12:17, 11 September 2008 (UTC)

The two main objections to this guideline regard a preference for secondary sources over primary and discouraging editors from using the popular press (newspapers, magazines, TV) as a source of medical facts. The first point has been queried by two editors, of which Paul is the most vocal. The latter has been mostly queried by Nbauman, a journalist.

• Both points have been part of these guidelines since since October 2006 when some text was adopted from WP:RS.
  • The warning "popular newspaper and magazine sources are generally not reliable sources for science and medicine articles" had been in WP:RS since April 2006 and remained until the page was rewritten/gutted in December 2006 when the text was moved to Wikipedia:Reliable source examples, an essay. Newspapers remain at the very bottom of the quality pile at the policy page WP:V: "In general, the most reliable sources are peer-reviewed journals and books published in university presses; usually followed by university-level textbooks; then by magazines, journals, and books published by respected publishing houses; then by mainstream newspapers."
  • The guidance "In general, Wikipedia articles should not depend on primary sources but rather on reliable secondary sources who have made careful use of the primary-source material." is similarly to be found in WP:RS from April 2006. It can also be found today in policy: WP:NOR says "Wikipedia articles should rely on reliable, published secondary sources."

These objections can be dismissed as in order to accomodate them, WP:MEDRS would have to be in conflict with both policy and long-standing guidelines that have wide consensus. Pleas to "revert to the last consensus version" are ridiculous since the only version that ever suggested newspapers might be OK at times, was one edited unilaterally and against consensus by Nbauman.

It should not be underestimated how damaging it would be if WP promoted the use of primary sources (primary research papers) or newspapers as a source for medical facts. The very worst of our POV pushers use such sources routinely because one can prove almost anything by carefully cherry picking basic research papers, or the uninformed scribblings that make up much press coverage of medical stories. Those editors with the educational and professional background that enables them to identify which newspaper articles are actually OK, or which primary research papers are worth citing, don't need WP:MEDRS to tell them how to write. And there's damn few of them. Colin°^Talk 13:13, 11 September 2008 (UTC)

• I oppose guideline status but can probably be ignored as an outlier because I fundamentally don't get how most points are specific to medicine (rather than scientific standards say, or existing core policies), or how it's in line with NPOV to subject a certain subset of pages to different basic standards/rewordings of core policies, and e.g. to say that if an article relates to medicine than medical sources are the "ideal sources" when infact sources and views from other fields are just as ideal according to the core policies. I guess this may relate to Paul Gene and others points about rewording core policies regarding primary/secondary sources. I didn't really want to get into this here but I thought I should just mention my take. EverSince (talk) 15:09, 11 September 2008 (UTC)

• I don't see that the guideline was promoted inappropriately. Consensus on Wikipedia is not unanimous support. There was broad support indicated in the discussion for promotion. This seems to be the general consensus of some of our best editors in the medical articles section of Wikipedia and I support their efforts to improve standards here. The guidelines should be useful to all editors - I certainly gained greater insight into what I shouldn't be doing by reading them. -- SiobhanHansa 20:35, 11 September 2008 (UTC)

A few observations:

• Wikipedia:Polling is not a substitute for discussion#Policy and guidelines guideline states "Wikipedia is not a democracy; policy and guidelines are not ratified through a vote", and WhatamIdoing observed at Wikipedia talk:Reliable sources#WP:MEDRS that " I could have just declared it a guideline -- but it's traditional at WPMED to make the extra effort to get community feedback in cases like this". So straw polling "Rfc Promotion" was not a requirement or essential, but helped foster a discussion that lasted 3 weeks about a proposal that had been 22 months in the drafting (10 November 2006 to 2 September 2008). By comparison WP:RS itself underwent 2.5 months of drafting and look how much that has changed since.
• Wikipedia:Policies and guidelines#WikiProjects policy notes re Wikiproject recommendations that "some are eventually considered guidelines after sufficient consensus has been reached", and "sufficient consensus" indicates that consensus is not necessarily a 100% agreement as SiobhanHansa correctly observes, but a value judgement on the level of consensus.
• To EverSince points, thank you - yes this is an "outlier" as it attempts to give further help on the main WP:RS (much as WP:MOS has several outlier sub-pages). As such there should be no implication of over-riding or rewriting WP:RS, just an attempt to help illustrate with examples (as guidelines do) and the "Assess the quality of evidence available" is a fairly international standard for biomedical journals and so useful to explain (it forms basis of generally prefering in the real-world review articles as secondary sources to single good quality studies, to observational studies, to annecdotal reports - but common sense and exceptions of course apply).
• As for the "contenious" bit on role of newspapers, that anyone is arguing this aspect indicates that at the very least the value of such sources is not necessarily straight forward and editorial judgement is required - for as the guideline header tag indicates: "should be treated with common sense and the occasional exception". Finally I fondly recall an article in The Times that criticised the confusion of the terms 'virus' and 'bacteria' in the popular media as giving false expectations for patients that antibitics could cure the common cold or sore throat, only to conclude that antibiotics were only of use for viral infections ! (go see "Common cold bacteria may cause blindness" which I wont treat as a reliable source, and Queen's Univerisity choice of press release title "Queen's researcher links common cold bacteria with cardiovascular disease" is misleading, although the report itself is well worded). David Ruben ^Talk 01:58, 12 September 2008 (UTC)

I should clarify that I meant my objection could be considered an outlier (partly 'cos I kind of tried to object to MEDMOS too, for similar reasons, though I ended up just confusing myself at the time). EverSince (talk) 02:59, 12 September 2008 (UTC)

I can also give examples of lies written in journals. For example, compare these two articles: [15] [16]; the author of the former has a political agenda. Another problem is its emphasis on scientific articles. Only doctors have access to lists of patients, therefore only doctors can do scientific studies. What about misdiagnosis? People who are forced to be subjects of drug trials against their will? Malpractice? Doctors will not write about these topics for obvious reasons. So this guideline can be used to silence anyone who would write about these topics. Only a few editors have supported this proposal and most of them are in the field. It's as if employees of tabacco companies voted that only tabacco companies publications are allowed on tabacco articles. --Mihai cartoaje (talk) 04:03, 14 September 2008 (UTC)

• Please quit wasting my time. Paul, you were on the losing end of the consensus. It happens. We've each individually been on the losing end of a consensus before (and probably will be again). Don't you have anything more important to do than to drag this conversation out for yet another week? Aren't there thousands of articles you could be writing now? This consensus is not going to change just because you're still unhappy that this guideline gives examples and explanations of how WP:PRIMARY applies to medicine-related articles. Alternatively, if you can't do anything else while this guideline conforms with a policy that you dislike, then please go over to [[[WT:NOR]] and talk them out of WP:PRIMARY. This guideline will not contradict Wikipedia policy, even if two editors disagree with certain aspects of the policy. WhatamIdoing (talk) 03:16, 12 September 2008 (UTC)

• • Dear fellow editors! I am not trying to waste your time. This is not a discussion about primary and secondary sources. This is not the rehash of the vote which happened a month ago. This is a discussion about whether administrators can promote essays to guidelines at their discretion when the consensus has not been reached. I specifically asked the opinion of the wider community, as other people may have a wider knowledge of such precedents. Unfortunately Rfc bot malfunctions. It assigned the wrong date to the discussion. It also developed a personality and instead of giving it the summary I gave it "MEDRS was promoted to guideline over the objections of a significant minority of editors." it gave the discussion the following inane summary: "The reliable sources page for medicine-related articles at WP:MEDRS is widely used and widely referenced on talk pages. WikiProject Medicine solicits comments on the page before assigning it [[Wikipedia:Policies and guidelines|guideline]] status. (11:39, 11 SeptemberOctober 2008 (UTC))" No wonder the opinion of the wider community has not been forthcoming. I am trying to contact the RFC bot owner. Paul Gene (talk) 11:16, 12 September 2008 (UTC)
    • Paul, you don't seriously think that a human reading that page is going to say, "Oops, that RFC is dated for a month from now, so I should ignore it," do you? The date listed is the date that the RFC is going to be removed, not the date that it was opened. WhatamIdoing (talk) 18:48, 12 September 2008 (UTC)

A consensus was reached. A consensus that a consensus was reached has been reached. --Arcadian (talk) 12:20, 12 September 2008 (UTC)

Well put. BTW: Any editor can create or demote a guideline. Admins aren't special in that regard. Colin°^Talk 12:44, 12 September 2008 (UTC)

Well put. A consensus has not been reached, I shall demote this essay. Paul Gene (talk) 10:01, 13 September 2008 (UTC)

The following comments by DavidRuben: "Ok I've acted as an admin and promoted from a proposed guideline to an actual guideline" [[17]] and "That said, admins weigh up proposals on the strength of the points raised, and frankly 2 of the opposing views were I thought weak and well argued against..." [[18]] seemed to suggest that only admins can do it. Sorry, it confused me. Paul Gene (talk) 10:58, 13 September 2008 (UTC)

I feel that I was double-crossed in this discussion. I discussed my changes in Talk before making them and tried to gain consensus. Other editors reverted my changes without discussing them first, and made their own controversial (and wrong) changes without discussing them in talk. I tried to do the right thing, and the collegial thing, and others took advantage of me for doing that. Nbauman (talk) 21:00, 12 September 2008 (UTC)

I can see the why Nbauman feels that way. I tried to approach this essay the same way I approach potential FAC articles. It is raw, contains a lot of nonsense and redundancies but can be worked on and made really useful. And all of a sudden, DavidRuben decided that he has consensus to promote this page to guideline, when it was not even nearly ready. That was unwise and rash decision. It promoted the feelings of bitterness and the division between editors who were on the "right" and "wrong" sides of the issues. It discouraged further discussions and improvement. How David should have handled it? I kinda imagine this now almost extinct on Wikipedia figure of a Wise Old Admin who would have said: "Children! Go back and work out your differences. Make this page better, and then we will talk again." :) Paul Gene (talk) 11:15, 13 September 2008 (UTC)

Your approach is utterly wrong and may explain your urgent desire to demote a useful guideline. Unlike FACs which have a great push of effort by relatively few editors and then ossify once promoted, a guideline is always a gradual evolutionary work in progress. The promotion from essay to guideline occurs when it is generally accepted to be in a good enough state to be useful. A few trolls, vandals and stubborn nit pickers (choose your own category) should not derail what has been agreed by a large number of not-inexperienced editors. Tell me, how will this guideline prevent you doing good work on WP? Do you really have to be so disruptive because it hasn't reached FAC levels of precision and quality prose? I expect your recent demotions to be swiftly reverted. Colin°^Talk 11:27, 13 September 2008 (UTC)

Dear Colin. My approach could be wrong but please do not punish me for being open with harsh words like "disruptive", "trolls, vandals and stubborn nit pickers (choose your own category)". There could be good sides in "stubborn nitpicking", look for example at SandyGeorgia's work. And there is a general concern about proliferation of various guidelines reflected, for example, in this discussion.[[19]] Maybe the time has come to be more judicious about promoting essays to guidelines? I certainly did not feel "urgent desire" to demote this assay which was promoted on September 1. I gave myself time to reflect, and asked DavidRuben about his reasons for doing that. On reflection, it appears to me to be an ill-considered move, which defied basic principles of Wikipedia and is harmful for that reason. I believe, there are also reasons why using this essay as guideline would do more harm than good. I will address them later after this particular dispute about this promotion has ended. Please remember these words from the banner on this page: "# Be polite # Assume good faith# No personal attacks". Paul Gene (talk) 12:24, 14 September 2008 (UTC)

I'm sorry that you feel bad about this, but let's make sure that we have the situation correctly summarized here:

• You proposed changes on this talk page.
• You valiantly attempted to convince people that your changes were positive/helpful/desirable, but were opposed at every turn.
• You made (some of) your proposed changes anyway.
• Other editors looked at the talk page, realized that there was not only no consensus to support your proposals, but actually firm opposition, and reverted your anti-consensus changes.

I'm not sure that your behavior here really exemplifies collegial effort. The process is not propose-attempt consensus-make changes. It's propose-IF consensus, THEN change, ELSE stop.

I can understand why you might feel like your views were ignored or even disrespected. I cannot understand why you think that removing your anti-consensus changes is a violation of consensus. WhatamIdoing (talk) 21:37, 12 September 2008 (UTC)

Point of fact: consensus is not reached because some editor (or even some group of editors) declares it is reached. that's an inappropriate and incorrect way to close a discussion. consensus is reached when all editors' concerns are addressed and at least minimally satisfied. if you neglect or refuse to enter into a discussion with other editors on certain points, then any claims of 'consensus' you make are specious and groundless, and can only be viewed as an attempt to disrupt the consensus process. clear enough? --Ludwigs2 21:44, 12 September 2008 (UTC)

Nope, I checked Wikipedia:Consensus, and I find not one word about requiring unanimous agreement.... despite, I might add, your attempts last month to redefine it as "all parties concerned with a particular article agree to the result of an edit or series of edits, or to the content of the article as a whole." I think you can safely assume that the firm opposition you encountered there is shared by a significant majority of editors here. WhatamIdoing (talk) 21:55, 12 September 2008 (UTC)

lol - and you find 'not one word' of it there, because you yourself have been adamant in removing it when I put it in. I get it now - this is an issue of page-ownership for you. you don't want consensus to actually be what most people would consider consensus, because that would limit your ability to unilaterally discard edits that disagree with your perspective. geez, you're a card, aintcha... --Ludwigs2 23:37, 12 September 2008 (UTC)

Actually, if you could be bothered to check your facts, I believe you will find that I've never edited WP:Consensus. WhatamIdoing (talk) 00:17, 13 September 2008 (UTC)

maybe not, but you sure have been arguing with me there. no matter, I opened up an RfC on it. let's see what the community says. --Ludwigs2 00:26, 13 September 2008 (UTC)

• The following parts of the WP:CONS policy were ignored when this essay was promoted to the guideline. Thus I have taken a decision to demote it.

"In the case of policies and guidelines, Wikipedia expects a higher standard of participation and consensus than on other pages." (bold is mine. PG)

"Developing consensus requires special attention to neutrality - remaining neutral in our actions in an effort to reach a compromise that everyone can agree on." (bold is mine. PG)

"Minority opinions typically reflect genuine concerns, and their (strict) logic may outweigh the "logic" (point of view) of the majority. New users who are not yet familiar with consensus should realize that polls (if held) are often more likely to be the start of a discussion rather than the end of one. (bold is mine. PG)

Paul Gene (talk) 11:37, 13 September 2008 (UTC)

Paul, rather than wikilawering and citing irrelevant policy (WP:NPOV does not apply to guidelines pages, for example), please move on and accept that your points do not have wide approval. You disagreed with this guideline on the sole reason that you disliked the preference for secondary sources. This preference is enshrined in guideline and policy elsewhere on WP (see above). Unless you have another strong reason to object to this guideline, further argument is likely to get you nowhere. People coming to this RFC will want to know why you are kicking up a fuss. If it is merely that you think procedure wasn't followed, it will be met with a huge shrug. Colin°^Talk 12:48, 13 September 2008 (UTC)

It's clear that we have overall consensus here, but not unanimity. This is enough for a guideline. For every guideline, some editors disagree with parts of that guideline. That does not mean every guideline should be demoted. I have therefore reverted Paul Gene's demotion, as not being a consensus change. Further discussion, along the lines that Paul Gene suggests, can occur here, independently of whether the page is a guideline. Also please see Wikipedia talk:Consensus #RfC: definition of consensus for an ongoing discussion that I expect was sparked by this thread. Eubulides (talk) 15:16, 13 September 2008 (UTC)

by contrast, there's no sense making a guideline that large numbers of editors will ignore as unsound. --Ludwigs2 21:41, 13 September 2008 (UTC)

Colin, it is bizarre to call WP:consensus policy irrelevant to this discussion. Unlike WP:NPOV it clearly applies to the whole body of Wikipedia (see above quotations). My "wikilawering" was made necessary by the assertions of Eubulides and WhatamIdoing (who invoked the WP:CONS policy) that consensus does not have to include everyone. To answer Eubulides. He, Colin, DavidRuben and others believe that the majority can ignore the opinions of several editors. They call that "consensus". While they have the majority here, their definition defies common understanding of consensus, and the understanding of consensus by the wider community as described in WP:CONS policy. WP:CONS specifically addresses such cases: "Consensus among a limited group of editors, at one place and time, cannot override community consensus on a wider scale." Paul Gene (talk) 11:42, 14 September 2008 (UTC)

I support this page as a guideline, even though I'm sorry the move was made, as it bought a problem for free. There is nothing significant in this guideline that isn't supported by WP:V policy, except some medical terms definitions and concepts that clarify things relative to medical articles. The page is a helpful guide for medical aricles, but significant editing practices won't change whether this page is a guideline or not, because the page correctly reflects WP:V policy. I suggest that editors arguing against policy might do that at WP:V. SandyGeorgia (Talk) 22:03, 13 September 2008 (UTC)

The way how the move was made is unfortunate. As thousands of successful examples of work on the featured articles candidates show, the consensus can be reached within few days if the editors are willing. This essay actually draws more on WP:NOR that WP:V, so editors are encouraged to participate in the discussions at WP:NOR as well. The editors who argue with the common definition of consensus are also welcome to take part in WP:CONS discussions. Paul Gene (talk) 12:58, 14 September 2008 (UTC)

This page (WP:MEDRS) is OK as an essay but would be bad as a guideline. It is excessively prescriptive and does not mention editorial discretion even once. The whole problem of primary vs secondary sources is confusing. While the distinction is clear for historical sciences, for natural sciences (such as medicine) it is very hard to pinpoint. If you read the above discussions you will see that primary source may contain pieces of secondary sources, secondary sources contain pieces of primary sources, press-releases are in the grey zone, and even journalism which in historical sciences solidly placed into primary sources may more often be considered secondary source for science.

To spare editors this confusion, we should delete primary-secondary definition and all the related discussion and leave it up to the WP:RS to argue. Instead, we would leave the distinction between sources based on their origin: peer-reviewed (reviews and research papers), books, journalism (popular press and popular science press), press-releases and other forms of communication. It would be rather the way WP:V does it. This will make WP:MEDRS shorter and more readable. Editorial discretion should be emphasized.

This will require a massive re-write and editing out large portions of the text, which the guideline status would interfere with. This will require a fresh start and flexibility on the part of editors who often argued here that something is good because it had been in this essay for years and reverted based on this argument. This will require a lot of good faith. Do we have it? Paul Gene (talk) 13:18, 14 September 2008 (UTC)

Paul, there is no reason that the content of this guideline can't be discussed and even radically changed while remaining a guideline. It happens on MoS quite regularly and happened to WP:RS a while back. If you want to work (with others) on an utterly revised version, it can be done on a sub-page and presented as an alternative for approval/discussion. Or done section by section.

The "consensus" dispute is a sideshow. Just troublemaking.

I really don't understand this desire to insert "use editor discretion" into individual guidelines. Editors have common sense (most of them), know the difference between guideline and policy, and we have WP:IAR. I find it patronising. It would be like the highway code telling me to drive on the left, except if there's a huge hole, a parked vehicle, a fallen tree, or a cow in the way.

The primary vs secondary issue is vital. Some editors find it confusing, some editors claim it is confusing (an entirely different thing!) and other editors hit the keyboard before engaging their brain. We must cover it and it really isn't that hard to pinpoint. Yes, individual works can contain a mixture of source types. So what. The historical sciences regard journalism as (mostly) a primary source because in the coverage of current affairs, the journalists are eyewitnesses. In the coverage of science and medicine, this is rarely the case. In those cases, the journalist is basing his writing on the writing of other (primary or secondary) sources. Is that so hard?

It is vital that we don't encourage editors to "perform their own review of the (primary) literature" as someone wrote earlier, as this makes it all too easy for the POV pushers or even well-meaning editors to mislead the reader.

I'm most disappointed that you are edit warring over the tag on this guideline. Removing the tag once was your BOLD statement. The second revert was just being disruptive. If your opinion is held by a significant number of other editors, couldn't you wait for one of them to act? Try WP:1RR? Colin°^Talk 14:53, 14 September 2008 (UTC)

• Oppose There is no consensus here, yet. Consensus is when there is unanimity in the sense that everyone involved finds the proposal acceptable (although some may still prefer another proposal). Stopping short of that is majority rule or fiat. I feel that this proposal is contrary to the intent if Wikipedia, in that it tries to prescribe what is fundamentally a matter of editorial judgment. If there is a content dispute, the dispute should be discussed and resolved via the dispute resolution process. This proposed guideline attempts to make rules to prevent disputes, and while the intention is good I think such rules are counterproductive. --Una Smith (talk) 14:54, 15 September 2008 (UTC)

  Please read WP:CONSENSUS: it has nothing to do with "unanimity". SandyGeorgia (Talk) 14:57, 15 September 2008 (UTC)

I have read it, and objected to it. It reflects the same misunderstanding about consensus that is echoed by some editors here. Someone here (I'm not sure who, signatures are lacking) wrote "The process is not propose-attempt consensus-make changes. It's propose-IF consensus, THEN change, ELSE stop." That editor is correct except instead of "stop", it should be "repeat". Many editors on Wikipedia seem to think a consensus process is "propose-attempt consensus-make changes", but they are mistaken. On Wikipedia, some good examples of a consensus process that works as it should can be found in formal mediation. There are two possible solutions to WP:CONSENSUS that I can accept: change the content, or change the title. We can choose merely to attempt consensus, but in that case we should not pretend that we are achieving consensus. --Una Smith (talk) 15:49, 15 September 2008 (UTC)

So, we have editors opposing the content of this guideline because it repeats aspects of policy they disagree with, and opposing the promotion of this guideline because the procedure followed aspects of policy they disagree with. If folk have problems with policy, discuss that on the policy page and accept that until those policy pages change, the rest of us should be given some slack in following them as best we can. Colin°^Talk 17:48, 15 September 2008 (UTC)

• Agree. If Wikipedia ever adopted a definition of consensus that required everybody to find the proposal acceptable then we may as well pack up and go home. Una, can you give an example where the guidelines would be "counterproductive". Colin°^Talk 15:10, 15 September 2008 (UTC)

The proposed guideline would be counterproductive by expanding content disputes into secondary disputes about the validity of the guidelines. Two disputes instead of one is not helpful. --Una Smith (talk) 15:49, 15 September 2008 (UTC)

But that is an argument against all guidelines whatsoever. Surely you don't favor repealing all guidelines? Assuming you don't, can you give a more-specific example where the guidelines in WP:MEDRS would be counterproductive? Eubulides (talk) 16:29, 15 September 2008 (UTC)

It is not an argument against all guidelines. It is an argument against guidelines that would use rules to shortcut the consensus process that is fundamental to Wikipedia. It tries to answer the question "which is the better evidence in this case?" by substituting for it (and answering) the question "which is the better class of evidence?" I have seen that substitution tried before, but I have not seen it work. --Una Smith (talk) 19:34, 15 September 2008 (UTC)

• Comment Since there's a couple of you who misunderstand WP:CONSENSUS, you are attempting to use to delete this guideline. Consensus is not unanimity. And it is not a loophole for one or two editors to block progress. Nevertheless, I'd like to know what we're trying to find consensus on? The group is in agreement that it's a guideline. Absent a good argument why it shouldn't be (and wikilawyering won't convince me, but what is specifically wrong with the guideline will), can we end this conversation. OrangeMarlin ^{Talk• Contributions} 16:39, 15 September 2008 (UTC)

I agree that an editor has declared it a guideline, and that some other editors claim the declaration reflects a consensus, and that there is a consensus in the sense that consensus is defined in WP:CONSENSUS. Where we don't agree is re whether the definition in WP:CONSENSUS is acceptable. That question is the subject of another ongoing RfC, by the way. --Una Smith (talk) 19:34, 15 September 2008 (UTC)

Colin above suggested that, if you strongly oppose this page being a guideline, you should remove the tag. If I am the only one doing it, the majority will continue to claim that the consensus exists and to ignore you with impunity. Paul Gene (talk) 01:59, 17 September 2008 (UTC)

I can't find where I suggested that and can't imagine why I would do so. I said "Any editor can create or demote a guideline". You could go and take the policy tag off WP:V but it would be very unwise to do so. It was not my intention to suggest or encourage any such action. Removing the tag is not the only way to make one's views heard, or to publicise a dispute. Colin°^Talk 08:47, 17 September 2008 (UTC)

Majority wants a war. Should the minority go along and give it to them? Colin, despite multiple "oppose" you incredibly claim that any kind of consensus exists. Your logics goes like this: "one editor is an outlier, another one should be dismissed for the extravagant views, third is a wikilawer, fourth - he is alone , we can ignore him. Hooray, we have a consensus because nobody reverts us!" You actually invite an edit war by writing to me: "Removing the tag once was your BOLD statement. The second revert was just being disruptive. If your opinion is held by a significant number of other editors, couldn't you wait for one of them to act?" Paul Gene (talk) 11:57, 17 September 2008 (UTC)

Paul, which discussion are you referring to? (1) the one that was held before David promoted the guideline or (2) this one on whether the promotion was valid. In (1) there were 18 participants and 15 support. One oppose was, it turns out, concerned with the idea that medically trained doctors might not be a reliable source for medical articles. I think we can all safely ignore that one. The other two (you and Una) objected to the preferential regard given to secondary sources. I really don't know what more can be said about that other than go and change the policy pages first, then come back here. In fact the same advice applies to the second discussion about consensus.

As for your interpretation of my words you quote: how is asking you not to continue making unilateral reverts an "invitation to edit war"? I'm seriously considering unwatching this page because discussing this further is not a productive use of my time. Colin°^Talk 15:36, 17 September 2008 (UTC)

Brandon Hill lists

This edit inserted two copies of recommendations for Brandon Hill Selected Lists. But Brandon Hill is no longer being maintained; the last version was issued in 2003. I prefer that the book list not be mentioned: many of the books on that list are obsolescent or even obsolete. The list of journals isn't so bad as it doesn't go stale so fast, but even there it's a bit iffy, and if it is mentioned we should caution in the main text that the list isn't being maintained. Eubulides (talk) 00:12, 14 September 2008 (UTC)

To be more precise, I propose removing the following text from WP:MEDRS#Medical journals:

Print Books and Journals for the Small Medical Library 2003 (Brandon Hill Selected Lists) lists 141 journals as a selection guide for health sciences library. Although the guide will no longer be updated , many of the journals listed have been included for many years in past guides.

and the following text from WP:MEDRS #Medical textbooks:

Print Books and Journals for the Small Medical Library 2003 (Brandon Hill Selected Lists) lists 672 books as a selection guide for health sciences library. Although the guide will no longer be updated , many of the books listed are updated frequently and a new edition may be available.

Eubulides (talk) 21:38, 14 September 2008 (UTC)

I support this proposal for the reasons given above. —Mattisse (Talk) 21:52, 14 September 2008 (UTC)

Is the list useful or not? Has anyone used it to choose which source to use, when PubMed or some other search turns up a selection? Would anyone really restrict their choice of sources in an article to just one book because that was the only one in this list? I can't see it helping me choose sources in the epilepsy-related articles. The goals of that list do not align with ours. It seeks to provide a restricted list of books and journals to a budget that would be useful in a small hospital library. We are not librarians but editors, and we will find and use the best sources that we can access -- which may be more or may be less than this list allows. Colin°^Talk 22:00, 14 September 2008 (UTC)

Myself, I would not use this list. I think the list exists primarily for people that need to find the "single best" source AND that don't know much, if anything, about the subject itself. By contrast, I'll use any reasonably current normal college or med school textbook that comes to hand, and not mind so much whether there's a slightly better textbook elsewhere. WhatamIdoing (talk) 05:12, 15 September 2008 (UTC)

Keep It is stated in the added text that the 2003 guide is no longer updated (reference included). The list is intended as a core collection ($144K); for libraries with a budget the list is marked with a * ($64K) or dagger ($18K). . "Journal title selections remain relatively stable." PubMed lists only journal articles which include primary and secondary sources. Books are secondary sources and are not listed in PubMed. The book list does include many of the standard textbooks used in medical school and by practicing physicians. The best books have been published for several editions; so a quick internet search for the newest edition should be easy. amazon.com also includes customer reviews e. g. Harrison's Principles of Internal Medicine There are usually more than one book listed for the different subject areas, so there is not a "single best" source, but several. Different medical schools use different textbooks for their classes, but many use the ones marked with a *. I added this list as a starting point for readers to see what books and journals are valuable. As a practicing physician and former assistant clinical professor, I've used this list for several years whenever I order a book for our clinic library. This list is only for those references in print; many now have on-line versions as well. I also use UpToDate which is only online and not included in the Brand/Hill print list. Petersam (talk) 09:23, 15 September 2008 (UTC)

Keep. I found out about the Brandon-Hill list when a medical librarian recommended it to me. I used it to find the major textbooks in each discipline, like (looking at my bookshelf) Harrison's internal medicine, DeVita's Cancer, etc. Not only do I use it to find textbooks to read, but since each of the authors is a recognized authority in the field, I can search PubMed for their articles.

Furthermore, even though the last version was updated in 2003, the selection has been relatively constant for decades; most of the textbooks have been recognized standards for generations. Harrison's is 50 years old. The textbooks are updated regularly, so the latest edition is not obsolete.

The journal titles are even more constant. Some of the standbys are over 100 years old. I use the Brandon-Hill list for searching PubMed. A PubMed search on a common topic can return thousands of hits. By selecting the major journals in the field, as defined by the Brandon-Hill list, I'm more likely to get important articles and reviews.

That's why it's useful. It's much better than nothing, and we don't have anything to replace it. Do we?

Citation analysis, while useful, is not the same. Citation analysis has commonly-acknowledged biases, for example, towards techniques. Citation analysis is generated by a computer algorithm, while the Brandon-Hill list was selcted by a team of librarians, most of whom have medical credentials themselves.

So the Brandon-Hill list is useful. I think the above answers all your objections. What objections do you have left? Nbauman (talk) 17:16, 15 September 2008 (UTC)

(outdent) There doesn't seem to be consensus for removing all mention of Brandon/Hill, but still, the current mention is way overkill. What I sense in the support for Brandon/Hill is nostalgia for the old way of researching medical topics, where one went to the library and read books. That was valid 20 or even 10 years ago, but nowadays almost everybody coming into the field reads this stuff online, and the Brandon/Hill list is (how shall I say it?) quaint. For more on this subject, please see the following source, which (I can't resist saying this) is freely available online: Ugaz AG, Resnick T (2008). "Assessing print and electronic use of reference/core medical textbooks". J Med Libr Assoc. 96 (2): 145–7. doi:10.3163/1536-5050.96.2.145. PMC 2268228. PMID 18379670.

With that in mind, I propose removing the paragraphs in question and then appending the following bullet to WP:MEDRS #Reliable references

• The Brandon/Hill selected list contains a list of books and journals suitable for a small medical library. Although the list is no longer maintained, its sources are high quality and many of the listed books are updated regularly.

This would of course come with a citation to the latest list. Eubulides (talk) 17:59, 15 September 2008 (UTC)

It's not nostalgia for the past, and the Brandon-Hill list is not quaint. Medical librarians, publishers and researchers have always eagerly embraced new technology, eg PubMed, and discarded obsolete technology, eg card catalogs.

The Brandon-Hill list was important because it captured the structure of how medical information is organized: There 'is' such a thing as core journals (141 of them, at one count) and textbooks (about 672 of them). People will be following that structure whether they're on paper or online.

The specific titles aren't all that important -- the important idea is that there is such a structure, and that a relatively small group of journals and books are accepted in the medical community as reliable.

I think the original wording is better than the proposed change. It's important to retain the idea of 141 journals and 672 textbooks, because it gives people a sense of the size and character of this body of knowledge. That's a manageable number of citations.

This is useful guidance for Wikipedia editors because editors are always doing Google or PubMed searches and coming up with obscure journals. It's important to explain to them that some journals are more important and reliable than others. It's one thing to see a study of an alternative medicine treatment in a small obscure publication that nobody ever heard of. It's another thing to see the same study in a major journal, like Urology.

I'll ask you: Do you have anything to replace the Brandon-Hill list, that explains the medical literature as well, and serves all of these purposes as well as the Brandon-Hill list does? Nbauman (talk) 19:39, 15 September 2008 (UTC)

Regardless of whether one considers a 5-year-old list "quaint", the latest proposal is not to delete mention of the list; it is to avoid covering that list too heavily. WP:MEDRS currently contains multiple instances of phrases like "core selection guide for health sciences library" and "although the guide will no longer be updated" and "high quality". There is no good reason for this duplication, particularly since most of the editors who've weighed in on this issue have expressed skepticism that Brandon/Hill should be mentioned at all. The only explicit objection you've given to the latest proposal is that it omits counts, so here's a new proposal for the bullet wording that fixes that problem, as well as mentioning the date of the list.

• The Brandon/Hill selected list mentions 672 books and 141 journals suitable for a small medical library. Although this 2003 list is no longer maintained, its sources are high quality and many of the listed books are updated regularly.

Eubulides (talk) 21:05, 15 September 2008 (UTC)

Petersam has followed up by making further edits, without discussing them here. These further edits do not address the main problem, which is that the text contains significant duplication and overemphasizes a list that many editors would like to see dropped. It would be better to discuss these changes before installing them. The above proposal (of removing the duplicative discussion, and replacing it with the bullet proposed above) still stands. Eubulides (talk) 22:51, 15 September 2008 (UTC)

Sorry, just incorporating some of the previous ideas to see how they looked....I shall wait... It's not necessarily a duplication. The first section on medical journals has an intro paragraph, then goes into a subsection on core journals. The Brandon Hill list on journals is relatively stable and includes a small core list of 59 journals because a library "usually cannot afford or may not need all of or most of the listed (141) publications." The article currently lists only 6 core journals. Rather than listing all the journals, having a reference to a list would be helpful. Maybe that paragraph should go below into the Core journal subsection. The section on medical textbooks does not have a subsection on core textbooks. Brandon Hill lists 672 books with 251 suggested for first purchase and 104 listed as a minimal core book list. These core books are high quality and are updated regularly...e.g. Gray's Anatomy 38th ed., Stedman's medical dictionary 27th ed., Rosen's Emergency Medicine 5th ed., Family Medicine 6th ed., Harrison's 15th ed., Washington Manual 30th ed., etc. So, the core list is also fairly stable; you just need to get the lastest edition. Instead of listing a bunch of the core textbooks, having a reference link is helpful. Again, this can be a leading to a new subsection on "Core Textbooks" similiar to the subsection on "Core Journals" listing only a few instead of many. Brandon Hill has been a valuable reference for 38 years. I'm sure some medical librarian somewhere will be continuing this resource. When I find it, I shall list it as an update. Any other thoughts? Petersam (talk) 00:41, 16 September 2008 (UTC)

The phrases I mentioned (e.g., "core selection guide") are indeed duplications within WP:MEDRS. The previous comment does not otherwise seem to address the specific proposal made, namely to coalesce these duplications and list Brandon/Hill just once. If some medical librarian somewhere updates Brandon/Hill sometime, that'd be fine, but in the meantime the current list is obsolescent and should not be emphasized so much here. Eubulides (talk) 03:14, 16 September 2008 (UTC)

I believe that this guideline should educate WP editors in the way that medical publication works, in particular the way the literature is divided into core journals and textbooks in general medicine and the specialties. The Brandon-Hill list is important because it teaches this idea, not just because it lists specific journals or books.

I don't think it's as useful to give WP editors a list of particular books that we have decided are "reliable". We should teach them how to arrive at that conclusion themselves. Nbauman (talk) 02:07, 16 September 2008 (UTC)

Nbauman wrote: The specific titles aren't all that important -- the important idea is that there is such a structure, and that a relatively small group of journals and books are accepted in the medical community as reliable

I disagree with this assertion. The only reason to have this list at all is to provide guidance on selecting specific titles. Yes, there are journals that have very good reputations and journals that have very... weak reputations. But I think that pretty much everyone knows that. We don't need this list to "prove" that some journals are considered better than others. It's not a disputed fact. The only value that this list provides is the list of specific, "approved" titles.

And while I don't actually mind the list of journals, fundamentally my problem is with appearing to recommend one perfectly decent textbook/reference book over another equally decent book. The fact that Brandon Hill, for example, chose Wintrobe's Clinical Hematology (ISBN 0781736501) to Turgeon's Clinical Hematology (ISBN 0781750075) doesn't mean that Turgeon's isn't a perfectly decent book. In fact, I'd go so far as to say that either of these books are better than any research publication in any journal. Their choice to exclude Turgeon's fundamentally means "A small medical library only needs one book like this," not that Turgeon's (or several other similar texts) is unreliable. WhatamIdoing (talk) 02:28, 16 September 2008 (UTC)

If you look at the Brandon Hill list for Hematology; you will find 4 minimal core books including Williams Hematology (ISBN 0071435913) , 7 initial purchase books, and 15 total books. Sure, Turgeon's may be a decent book, it's just not as comprehensive as the other two. Got Doody's rating by emailing a local medical reference librarian in Oakland; they are very helpful in helping anyone find information. The Doody Hematology book ratings were by Content Specialist Steven Rosen, MD [20], Northwestern University Feinberg School of Medicine, Robert H. Lurie Cancer Center, Chicago, IL plus 2 Librarian Selectors [21].

Criteria	Wintrobe's	Turgeon's	Williams
Edition	11th	4th	7th
Pages	2800	570 (20% of Wintrobe's)	1856
Price	$239 ($191)	$92 ($74)	$215 ($172)
Brandon Hill	minimal core list	not listed	minimal core list
Doody's score Lab Med 0-3	2.9	-
Doody's score Hem/Onc 0-3	3	2.4
Doody's Star rating 0-5	5	4

. Petersam (talk) 07:31, 17 September 2008 (UTC)

---

Found a replacement for the Brandon Hill books with Doody's Core Titles. Seems to be consensus to keep Brandon Hill for journals. Petersam (talk) 08:01, 16 September 2008 (UTC)

• There is no consensus to keep. Two editors are in favor (you and Nbauman), two opposed (myself and WhatamIdoing), and one skeptical (Colin). I see that edits are continuing to take place in the project page without discussion in advance, and without responses to my suggestions here.
• Doody's Core Titles is not freely available. It is not a particularly good source for Wikipedia editors. It should not be emphasized on this page.
• For now I trimmed down the mention of Brandon/Hill and Doody's, as the previous text was clearly excessive.
• It is still not at all clear that Brandon/Hill and Doody's should be mentioned here. It would be more convincing if we knew that Wikipedia editors of medical articles were actually using Doody's and Brandon/Hill so extensively that they were worth citing here.

Eubulides (talk) 16:53, 16 September 2008 (UTC)

How about posting a query at WT:MED to see if anyone uses them. Colin°^Talk 17:11, 16 September 2008 (UTC)

Thanks for the suggestion. I asked at WT:MED#Brandon/Hill and Doody's?. Eubulides (talk) 18:11, 16 September 2008 (UTC)

I don't know why you're trying so hard to remove information that some of us think is useful. You raised the objection that the list is obsolete, and we answered it -- it's not obsolete. You've given no reason for removing it except WP:IDONTLIKEIT.

There is no consensus to keep, but there is no consensus to delete, and -- to use your own argument -- it's been in the guideline for about a year. Nbauman (talk) 18:36, 16 September 2008 (UTC)

• (ec) I very strongly agree with User:Nbauman's comment above: I don't think it's as useful to give WP editors a list of particular books that we have decided are "reliable". We should teach them how to arrive at that conclusion themselves. Providing a list of preferred sources from 2003 does just the opposition. It gives the uninformed editor the impression that these sources are reliable sources for whatever article that editor is sourcing. —Mattisse (Talk) 18:38, 16 September 2008 (UTC)

As we explained above, this is a 2003 collection of books and journals that have been recommended by authoritative sources for decades. Every title on that list is still a major journal or book today.

They are reliable sources. Can you cite one title on that list that is not reliable? Nbauman (talk) 19:57, 16 September 2008 (UTC)

They may be dated or lacking in information on certain subjects, for example the posthumorous diagnosis of Samuel Johnson with Tourette syndrome. It is not that a particular book is globally unreliable. Although a book that has not been updated since 2003 might contain outdated information. That is a long time ago in the world of scientific research. Are you saying that they are all regularly updated? —Mattisse (Talk) 20:26, 16 September 2008 (UTC)

Journals are not reliable sources. Journal articles are. Certainly one can cite unreliable articles taken from a Brandon/Hill-listed journal (one example: Wakefield et al. 1998, PMID 9500320). But this is straying from the point; please see #Brandon/Hill and Doody's are not that useful for Wikipedia below. Eubulides (talk) 20:30, 16 September 2008 (UTC)

Almost. Journal articles, properly used, are reliable sources for Wikipedia's purposes. If you've ever watched RSN for any length of time, you know that reliability can't be determined in a vacuum. A perfectly reliable source for fact X may be hopelessly unreliable for fact Y. WhatamIdoing (talk) 04:11, 17 September 2008 (UTC)

Brandon/Hill and Doody's are not that useful for Wikipedia

• Brandon/Hill omits high-quality journals like Cochrane Database Syst Rev that are too new to make the list. I just now checked all uses of {{cite journal}} in Wikipedia's featured medical articles. The #1 journal, with 50 citations, was N Engl J Med (not surprisingly). But #2 was Cochrane, with 48 citations, and Cochrane is too new to be on the Brandon/Hill list.

Be careful of your conclusions -- Cochrane was not in the Brandon/Hill Selected List of Print Books and Journals for the Small Medical Library because Brandon Hill only covers PRINT media, not electronic databases like the Cochrane (see bottom of page 2. Petersam (talk) 07:42, 17 September 2008 (UTC)

This underscores the obsolescence of Brandon/Hill. Its page-2 mention of "audiovisuals or microcomputer software" is a charming anachronism. Even in 2003 it was strange and old-fashioned to exclude discussion of online access, with regards to a medical library, as the vast majority of research medical journals were searched for and accessed online. Nowadays, of course, medicine is even more online than it was 5 years ago. Again, there's no indication here that Brandon/Hill is all that relevant to Wikipedia editors today. (And for what it's worth, you can order printed copies of individual Cochrane reviews from the publisher, though I expect hardly anybody does that.) Eubulides (talk) 08:19, 17 September 2008 (UTC)

• And this is not the only sign that Brandon/Hill is obsolescent. For example, J Infect Dis was a reasonable choice ten years ago for a journal on infectious disease, but these days Lancet Infect Dis publishes more high-impact papers in that area. And the list's coverage of psychiatry is antediluvian: it doesn't list any journal on the topics of autism or schizophrenia, for example. If Autism had been written by editors relying on Brandon/Hill, it would be a much, much weaker article.
• Doody's does not suffer from Brandon/Hill's problem of obsolescence; but it is a proprietary source that Wikipedia editors by and large do not have access to.
• These objectious would be overcome if there were evidence that the use of Brandon/Hill or Doody's was improving Wikipedia articles. So far we haven't seen any evidence of that. My own impression is that mentioning Brandon/Hill and Doody's here is not helpful enough to justify their inclusion.

Eubulides (talk) 20:30, 16 September 2008 (UTC)

Could you put the list you generated into a public space? Also, I've created a stub for Brandon-Hill list, and a stub for Doody's would also probably be a good idea. --Arcadian (talk) 21:36, 16 September 2008 (UTC)

I put the list this version of my sandbox. Eubulides (talk) 22:05, 16 September 2008 (UTC)

Press review websites

The text currently says

Sources for evaluating health-care media coverage include review websites such as Health News Review and specialized academic journals such as the Journal of Health Communication; reviews can also appear in the American Journal of Public Health, the Columbia Journalism Review, The Guardian, and others.

I think it would be more useful to editors if the Health News Review was a direct link to the web site. I would like to add the UK's Behind the Headlines which does a good job of staying on top of the latest headlines and links to the scientific papers. In addition, Ben Goldacre's column in The Guardian would be more usefully linked as Bad Science since otherwise an unfamiliar reader wouldn't know where to look in that paper's huge archive. Colin°^Talk 20:19, 16 September 2008 (UTC)

These suggestions all sound good to me. I assume the wikilink to Health News Review would be kept? It's also useful. Eubulides (talk) 20:30, 16 September 2008 (UTC)

If you can find a way to keep the wikilink, OK. Look at WP:MOS's "Other resources" section. It doesn't bother with footnotes and just embeds the hyperlinks. Colin°^Talk 21:49, 16 September 2008 (UTC)

OK, I gave it a shot with this edit. This also mentions Media Doctor, a pioneer in this field. Eubulides (talk) 22:27, 16 September 2008 (UTC)