Wikipedia talk:Identifying reliable sources (medicine)

Medicine and the popular press

An interesting and relevant article from the New England Journal of Medicine on the difficulties and pitfalls inherent in popular-press coverage of science and medicine is here. I thought it might be relevant, and perhaps even useful as a source. MastCell ^Talk 20:34, 8 January 2009 (UTC)

Thanks, I added the phrase "news articles too often convey wrong or misleading information about health care" and cited that new source. Eubulides (talk) 21:14, 8 January 2009 (UTC)

I think "too often" is an exaggeration based on the anecdotal evidence in that article (he cites a handful of examples). So I'm changing it to "sometimes". Xasodfuih (talk) 11:36, 24 February 2009 (UTC)

He's slamming WaPo for instance for saying something along the lines "antidepressants don't work half the time", when he thinks that the fact that they do work half the time a great achievement, so in his view WaPo should have written the article with that spin. Spin on spin. Xasodfuih (talk) 11:39, 24 February 2009 (UTC)

Take a look at the current front page of Health News Review; it cites a recent ABC News story on "cooking" cancers, a recent ABC News story on platelet-rich plasma, and a recent CBS News story on lunchtime liposuction, all published February 16 or later, and all terribly misleading or wrong. It's quite bad in today's journalism, at least in the English-language press. Anyway, the cited source says "all too frequently" and it's straying to far from that to change it to "sometimes"; I reworded it at "all too often". Eubulides (talk) 18:25, 24 February 2009 (UTC)

"All too often" is a meaningless term. In my experience, it's a term that people use when they don't have accurate figure. It's non-falsifiable. They could say that once is too often. In a book, magazine, or legal writing, editors strike out terms like that and send the writer back to get the actual figures.

How often is "all too often"? What's the difference in meaning between "too often" and "all too often"? Is once "too often"? More than half? 20%?

All too often, this MOS is nonsense.

It's bad writing, because it's meaningless. In WP terms, it seems to violate WP:WEASEL and WP:PEACOCK. Nbauman (talk) 18:49, 24 February 2009 (UTC)

The phrase in the cited article is "all too frequently". The editors of the NEJM didn't strike it out and "send the writer back to get the actual figures". The word "too" is the key and is a matter of opinion which can't be derived from raw data. In the opinion of the NEJM article writer, and most of the contributors to this page, journalists say "wrong or misleading" things about health too often for us to consider them generally reliable. Sorry if that offends you but that's our opinion. Wrt WEASEL/PEACOCK, WP guidelines do not have to follow WP guidelines or policy (none of them do). Colin°^Talk 19:06, 24 February 2009 (UTC)

I read that Dentzer article when it came out in the NEJM, and I didn't think it was particularly useful -- just a lot of de gustibus judgments about how she disagreed with the way the press handled particular stories. I didn't think the Jim Lehrer NewsHour coverage of health care policy was that good, as others have argued [citations omitted].

You could just as easily say that the NEJM gets its peer-reviewed articles wrong "all too frequently". Even 1 is too frequently. You could say that any publication makes mistakes "all too frequently". You could say that doctors make mistakes or commit crimes "all too frequently". Therefore, the phrase is nonsense. If you want to write nonsense, go ahead.

It doesn't offend me that most of the contributors to this page believe that journalists say "wrong or misleading" things about health too often for you to consider them "generally reliable."

It does offend me that you say that without any supporting evidence, evidence of the quality that you would expect others to use -- evidence in peer-reviewed journals, academic studies, or reviews of those studies. Indeed, you say that despite contrary evidence.

Instead, you say that on the basis only of your own personal opinions and prejudices, against newspapers and against things that ordinary people read.

This entry makes broad, sweeping generalizations attacking newspapers, contrary to the evidence in the peer-reviewed literature. And you say that you can do that because WP guidelines are merely opinion, and don't have to be supported by WP:RS or anything else, including facts -- as long as it's agreed on by a majority of the editors working on the guideline.

If you want to ignore the scientific method that you profess to champion, go ahead. Nbauman (talk) 21:19, 24 February 2009 (UTC)

Please can you cite the study in the "peer-reviewed literature" that concludes that newspapers are consistently accurate in their reporting of health issues? We need "consistent accuracy" for a type of source to be judged "reliable", not the odd example of outstanding journalism. My "prejudices" are amply confirmed by reading NHS Choices. Colin°^Talk 21:36, 24 February 2009 (UTC)

• Like Colin, I don't know of any "contrary evidence" on this issue; that is, I don't know of any reliable source that contradicts our reliable sources on this particular point.
• I would favor changing "all too often" back to "too often"; the "all" isn't really needed here and is I think part of the problem. However, changing "too often" to "sometimes" would go too far from what is in the source, Dentzer 2009 (PMID 19118299).
• "Too often" means too often to be a reliable source in the WP:MEDRS sense. I think this is reasonably obvious in context, but if not, I suppose we could insert further text to clarify this.
• It's not just this source; for example, Schwitzer 2008 (PMID 18507496) writes that U.S. news articles are "often ... imbalanced" and that "We believe this imbalance may have a profound impact on the decision making of American consumers who rely on these news stories." Schwitzer goes on to say that "People may be misled, become anxious, or make ill-informed decisions based on" the kind of incomplete stories that often are published by news organizations.

Eubulides (talk) 21:47, 24 February 2009 (UTC)

(Undent) Nbauman, you write professionally. Think about the last few science-related pieces you've done. About how often did you, in your professional writing, rely entirely on what some other journalist wrote for a scientific/medical fact, without bothering to check it against a journal article, textbook, or other source? Never? Always? Something in between? WhatamIdoing (talk) 22:30, 24 February 2009 (UTC)

In my writings, I have never relied upon another journalist. My job is basically to check the facts with the doctors who did the original research, and confirm with them that I understand their work correctly. And I would never depend on my own understanding of a published source. I would talk to the author to make sure I got it right. Many news stories, and many WP articles, are misunderstandings of published sources.

Some of you here may be medical doctors or its equivalent, but you don't understand journalism, and particularly medical journalism, at all. You don't know how it works. I wonder if you've ever spoken to a journalist about medical coverage, or called a journalist to complain when you saw a story that was wrong. I doubt it, because you're engaging in prejudiced stereotyping.

There are techniques that some journalists, and particularly medical journalists, use to make sure they get their facts right.

The important point that you're missing is that there is a great variation in journalism, some of it reliable, some of it not reliable, and there are ways of distinguishing between them. That's what Schwitzer studies. This guideline completely brushes that research aside, and doesn't give any guidance on how to distinguish them. You're choosing ignorance. I can't stop you. Nbauman (talk) 23:38, 24 February 2009 (UTC)

So you always reject a journalist's summary or interpretation for this purpose in your professional work, but you think that we should encourage editors to use a lower standard than the one you yourself use. A newspaper article isn't a good enough source for scientific facts for you, but you keep arguing that it ought to be good enough for Wikipedia, right? WhatamIdoing (talk) 05:12, 26 February 2009 (UTC)

That sounds like a trick question, which you could have used against me no matter what I answered.

No, your interpretation is wrong.

First, I said that I would verify my understanding of any published source -- newspaper or peer-reviewed article -- with the original investigator.

I have a good sense of how accurate newspaper stories are, because I often use newspaper stories as part of my research, and when I talk to the investigator I ask him whether the newspaper story about his work was correct. Some are, some aren't. Some newspapers, like the Wall Street Journal, have a widespread reputation for accuracy. Doctors themselves read the New York Times. I've been amazed to find out sometimes that even leading investigators had read the NYT article, but not the original journal article, about something important in their own field.

That's why I say you're ignoring the data. You haven't spoken to investigators and asked them whether they thought the newspaper coverage of their work was accurate. You would find out that some publications are accurate and some aren't. Instead, you're cherry-picking the newspaper stories and opinion pieces that fit your prejudices.

Second, the standard of accuracy that I use for my published stories is very high, because the fact-checking is my responsibility alone. (And I get paid for it.) After I publish it, I can't correct it any more.

The standard of accuracy that everyone uses on WP is not that high, because other editors will review the changes after they're put on the Internet and correct it if necessary. That's the whole idea behind WP.

That's all consistent with my main point: There is good scientific data, published by authors like Schwitzer, in the academic peer-reviewed journals, the standard you profess to follow, that newspaper accuracy is more complex than you claim it is, and that many newspapers cover medicine very well. And yet you refuse to follow those academic standards in this MOS, but instead insert your own unfair prejudices against newspapers. Nbauman (talk) 06:51, 26 February 2009 (UTC)

This guideline does not say that all newspapers and magazines get everything wrong every time, and where it dings popular press is for widely agreed deficiencies, such as reporting risks out of context.

As I'm sure I have mentioned several times during these ongoing conversations, I have spoken with investigators on this subject. I know a few that simply refuse all interview requests after getting burned with egregious errors (like "We have isolated a new compound that might have interesting biological effects" being turned into "New drug will be on the market soon"): They just ignore requests and direct the occasional persistent person to the publicity office. I know more who will only answer questions in writing. I even know one person that agreed to an interview on condition of proofing the story: She corrected the proof, and then had serious factual errors introduced by an editor. Of the people that have had their work written up in three or more popular press articles, I know no one that says it was adequately and accurately represented in all of them.

Presumably these are the people that aren't returning your phone calls, so they're not in your sample set.

Wikipedia should aim higher than just getting most of the facts right most of the time. We should have a high standard of accuracy. There's no harm in our recommendation that people get it right the first time. WhatamIdoing (talk) 19:48, 26 February 2009 (UTC)

Some science journalism is accurate some is not. Before adding info to wikipedia one should verify the accuracy to a peer reviewed source. Up in the cold cold north we play a game were you site in a circle and whisper from one person to next to see what comes back to you. This recreates the accuracy of moving farther from the source.--Doc James (talk · contribs · email) 20:29, 26 February 2009 (UTC)

I finally got a chance

I finally got a chance to read Susan Dentzer's NEJM piece carefully. I realize now that WP:MEDMOS distorts Denzer's article through selective misquotation. MEDMOS says

news articles too often convey wrong or misleading information about health care.

Dentzer actually wrote

Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading.

In other words, Dentzer said what I've repeatedly said, that (1) some popular news sources are accurate, while (2) others are not. MEDMOS selectively paraphrases Dentzer to make it sound as if she made a one-sided statement supporting (2), even though she actually made a balanced statement.

So here on WP we have critics of the popular press selectively quoting and distorting a source, to make the point that they're trying to push -- that the popular press is unreliable -- rather than accurately reporting what the source herself actually believes -- that the popular press is diverse.

I always take it seriously when people criticize the press, because I have an obligation to find out if I'm doing things wrong and to correct my procedures if I am.

However, critics often charge the press with the very errors that they, ironically, are committing themselves. That seems to be the case here -- cherry-picking one sentence in an article, while ignoring the other.

I will be interested to see whether you keep that distortion of Dentzer in the entry, or whether you change it. Nbauman (talk) 02:35, 24 March 2009 (UTC)

The preceding comment greately misrepresents the main thrust of Dentzer 2009 (PMID 19118299), by selectively quoting one of the few positive things it says about the popular press. The bulk of Dentzer's piece is critical of the press: this includes not only its title "Communicating Medical News — Pitfalls of Health Care Journalism", but also the topics of most of its paragraphs, ranging from "The news media need to become more knowledgeable and to embrace more fully our role in delivering to the public accurate, complete, and balanced messages about health." to "More broadly, a problem that is worsening in this era of the 24/7 news cycle is the frequent failure to put new developments into any kind of reasonable context for readers or viewers.". The piece's concluding two paragraphs lead with "In my view, we in the news media have a responsibility to hold ourselves to higher standards if there is any chance that doctors and patients will act on the basis of our reporting.", which pretty much sums up Dentzer's opinion. The existing summary in WP:MEDMOS is fairly accurate, but like any brief summary, it is not perfect: if you can propose better (brief) wording that does not misrepresent Dentzer's piece, please do so here. Eubulides (talk) 06:45, 24 March 2009 (UTC)

I "selectively quoted" from the introductory paragraph of the article. The introductory paragraph is where writers state the main thrust of their argument clearly to frame and qualify what follows. Dentzer states at the beginning that often messages are delivered effectively, but often they are not. The article deals with examples in which they were not. Nowhere in the article does Dentzer say that her examples characterize all medical stories, or medical journalism in general.

Her argument is that bad stories are done because some reporters don't understand medicine. She acknowledges in the introduction that some writers do understand medicine.

If you're going to quote Dentzer, you should quote what she actually said -- the positive and negative. If we can't agree on a summary, then we should use the full quote:

Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading.

I would paraphrase it as:

Some articles in the popular press, particularly those by experienced reporters, are accurate, while others are wrong or misleading.

That's what she said -- positive and negative. To use the negative, and ignore the positive, is wrong and misleading. Nbauman (talk) 15:45, 24 March 2009 (UTC)

I think that our advice lines up with the main thrust of Dentzer's piece, but I also don't think it much matters. The advice from Wikipedia editors, to other Wikipedia editors, is still the same: avoid using the popular press as your sole source for a medical or scientific fact. Our advice would be the same even if Dentzer had never written that piece.

If you ever find a situation where our deprecation of sole reliance on the popular press actually prevents you from writing a good article, please let us know. Until then, even if it hurts your professional pride to have known limitations called out without a placating admission that not every journalist gets every fact wrong every time, the advice about preferring the scientific literature to the popular press stands. WhatamIdoing (talk) 19:34, 24 March 2009 (UTC)

If you are attributing a statement to Dentzer, then it does matter whether your statement "lines up" with the source or not. The question is whether you want to use a misleading summary or not. You are insisting on using a misleading summary in order to advance your personal prejudices about the press.

Some people say that what matters on Wikipedia is not truth, but verification.

Now you're saying that neither truth nor verification matter. I want to get that on the record. This discussion is getting too serious, and we need some entertainment.

(BTW, you're misrepresenting my position too (Straw man). I never said that anyone should use the popular press as the sole source. And I never said that every journalist gets every fact right every time; I cited Schwitzer.) Nbauman (talk) 22:21, 24 March 2009 (UTC)

The proposed text is not an improvement on what's already in WP:MEDRS, as it less-accurately summarizes Dentzer 2009 (PMID 19118299) than the current text does. Dentzer's primary thrust, from its title to its conclusion (which is in its last 2 paragraphs), focuses on limitations and drawbacks of the popular press when reporting medical issues. The proposed text contains 13 words on the strengths of the popular press, and 6 words on its weaknesses, which is a seriously out-of-whack characterization of Dentzer from a WP:WEIGHT point of view. I'm not saying that the current text in WP:MEDRS can't be improved (far from it) but it's not an improvement to replace it with such a misleading summary. Perhaps you could give it another try, which summarizes Dentzer more fairly? 04:58, 25 March 2009 (UTC)

The subject of Denzer's article is specialist vs. generalist medical journalists.

Dentzer makes a standard argument, which I've heard many times among science and medical journalists, which is that specialist medical journalists can cover medicine better than generalist journalists.

Dentzer says in the introduction that specialist ("seasoned") journalists can deliver the messages well. She then goes on to give examples of non-specialist or poorly-trained journalists who (she believes) got the story wrong.

In other words, she believes that some journalists (specialists) are reliable, and others (poorly trained) are not.

The editors of this guideline believe something different. They believe that the popular press is "generally" not reliable. That is not what Dentzer said. Dentzer said what I said, and what Schwitzer said, which is that some journalists in the popular press are reliable and some are not.

Any paraphrase or citation of Dentzer's article, that supports your believe that the popular press is "generally" not reliable, is a deceptive misquotation or misattribution of Dentzer.

Any reference to Dentzer must preserve the idea that some journalists in the popular press are reliable, and some are not.

I think it's ironic that the people who fault the popular press for not reporting the medical literature reliably, are themselves deliberately distorting the medical literature to support their own prejudices and attribute a view to the author that she doesn't state or believe. But that's what makes Wikipedia so entertaining. Nbauman (talk) 17:07, 25 March 2009 (UTC)

Please see #Popular press is generally not reliable below. Eubulides (talk) 17:55, 25 March 2009 (UTC)

Popular press is generally not reliable

• "Dentzer makes a standard argument ... that specialist medical journalists can cover medicine better than generalist journalists." This is a misleading summary of Dentzer. Dentzer never says either "specialist" or "generalist", or anything like that. Her main point is news articles' "frequent failure to put new developments into any kind of reasonable context for readers or viewers". She also mentions lack of training, but that is secondary, and she never argues for specialist (as opposed to generalist) reporters.
• "The editors of this guideline believe something different. They believe that the popular press is "generally" not reliable." This is pretty much what Dentzer 2009 (PMID 19118299) wrote (albeit more politely, which is appropriate for the NEJM). Here's Dentzer's main conclusion:

"In my view, we in the news media have a responsibility to hold ourselves to higher standards if there is any chance that doctors and patients will act on the basis of our reporting. We are not clinicians, but we must be more than carnival barkers; we must be credible health communicators more interested in conveying clear, actionable health information to the public than carrying out our other agendas. There is strong evidence that many journalists agree — and in particular, consider themselves poorly trained to understand medical studies and statistics.(Voss 2002, PMID 12084701) But not only should our profession demand better training of health journalists, it should also require that health stories, rather than being rendered in black and white, use all the grays on the palette to paint a comprehensive picture of inevitably complex realties."

• Again, I'm not saying the current summary of Dentzer can't be improved on, but it summarizes Dentzer far more accurately and fairly than the proposed replacement. Please try to think of a better proposal.

Eubulides (talk) 17:55, 25 March 2009 (UTC)

You said,

Dentzer never says either "specialist" or "generalist", or anything like that.

Dentzer said:

Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines.

What do you think "seasoned reporters" means? Nbauman (talk) 18:51, 25 March 2009 (UTC)

I think it means "experienced", and so does the dictionary. See definition 2b. Experience and training are different. Dentzer believes that a veteran reporter that has spent years reporting all kinds of health stories will do better than a newbie that took one class on the specific subject. WhatamIdoing (talk) 20:04, 25 March 2009 (UTC)

I was asking Eubulides. Nbauman (talk) 20:48, 25 March 2009 (UTC)

I agree with WhatamIdoing that "seasoned" means "experienced", not "specialist". But isn't this a side issue? Eubulides (talk) 21:44, 25 March 2009 (UTC)

It's not a side issue, it's a factual distortion of a source.

Do you agree, then, that Dentzer is saying:

Often these messages are delivered effectively by experienced reporters.

Nbauman (talk) 00:00, 26 March 2009 (UTC)

Certainly it's a side issue, as far as Dentzer is concerned. The main "factual distortion of a source" that is going on in this subthread is the overemphasis on Dentzer's pro forma concession that news articles often get it right. But that concession is not Dentzer's main point; it's not even a secondary topic in her piece. And it should not be the main point here, where we have many more fish to fry than Dentzer's relatively narrow topic. It's not at all clear that it should be mentioned here at all. Eubulides (talk) 01:51, 26 March 2009 (UTC)

Eubulides, please answer my question. Do you agree that Dentzer is saying:

Often these messages are delivered effectively by experienced reporters.

Nbauman (talk) 02:47, 26 March 2009 (UTC)

I don't agree that that is Dentzer's main or even secondary point, no. It is true that Dentzer briefly concedes that point before making her main arguments about the relatively low quality of news articles as sources for medical information. Here's what Dentzer writes:

"Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading. Some distortion is attributable to ignorance or an inability to interpret and convey the nuanced results of clinical studies. And some is due to uncertainty about journalists' proper role: Is our job to describe the bigger picture, or simply to report what is "new"? Should we present black-and-white versions of reality that lend themselves to stark headlines, rather than grayer complexities that are harder to distill into simple truths? ¶ I believe that when journalists ignore complexities or fail to provide context, the public health messages they convey are inevitably inadequate or distorted. The news media need to become more knowledgeable and to embrace more fully our role in delivering to the public accurate, complete, and balanced messages about health. With some additional skills, care, and introspection — and a change in priorities — we can produce coverage more in line with our responsibilities."

Dentzer goes on to give multiple examples of news articles getting it wrong and explaining why this occurs. Overall, it would be misleading to summarize her comments with "Some articles in the popular press, particularly those by experienced reporters, are accurate, while others are wrong or misleading.", which is the text most-recently proposed. Eubulides (talk) 03:13, 26 March 2009 (UTC)

What about Although some articles in the popular press are accurate, many others are inadequate, misleading, or even completely wrong. Tim Vickers (talk) 03:31, 26 March 2009 (UTC)

Yes, that's better than what we've got now.

But I think we should also help Wikipedia editors to figure out which popular press articles are likely to be accurate and which are likely to be wrong. Dentzer made the point that experienced medical journalists are often accurate. I think that point also belongs in the guideline. Don't you agree? Nbauman (talk) 03:43, 26 March 2009 (UTC)

Interesting idea, how would you do this in practice? For example, how experienced is the author of this story? Tim Vickers (talk) 04:16, 26 March 2009 (UTC)

I think that asking the editor to divine the reporter's level of (relevant) experience is asking too much.

Tim, the problem with your proposal is that a statement can be technically "accurate" while also being "inadequate" and consequently "misleading". WhatamIdoing (talk) 04:30, 26 March 2009 (UTC)

Tim, in practice, the way to help WP editors figure out which popular press articles are accurate or wrong is to link to the many free web sites that have been set up by collaborations of journalists, social scientists, doctors and academics for that purpose, such as this and this. These include evidence-based, peer-reviewed studies of the press coverage of medicine.

I wanted to do that, but I was outvoted by editors who insisted that the entire popular press was unreliable. If you start by saying that the entire popular press is unreliable, then you can't go on to explain how some popular press sources are reliable and others are unreliable.

In the case of the Bloomberg story on tuberculosis, you could start by rating it on these criteria. There are other ways of rating it. There are many ways of rating news stories, and they've been discussed extensively in the peer-reviewed medical literature, in BMJ and elsewhere.

You can tell the reporter's level of skill by how well he meets these criteria. (Actually, what matters is not his experience, but whether he writes an accurate story.) These criteria also help you figure out whether the article is adequate or misleading. Nbauman (talk) 11:36, 26 March 2009 (UTC)

Two of those seem useful links, particularly in light of the fact that we already advise people to use the "laysummary" parameter in cite journal. They could help with selecting articles for those so I've made an addition to the guideline. Do you have any other specific suggestions Nbauman? Tim Vickers (talk) 15:37, 26 March 2009 (UTC)

TimVickers, My other suggestion is that we include language in the guideline to explain to people the repeated findings from the peer-reviewed literature about how the reliability of the popular press is distributed:

Some popular news publications, and individual journalists, are reliable, while others are not. Science and medical reporters in major publications, such as Lawrence Altman of the New York Times, are usually reliable. Reporters who don't specialize in science or medicine, and reporters who must write quickly on tight deadlines, are less reliable. No source is always reliable, and Wikipedia articles should be checked against the original published research or peer-reviewed review articles for accuracy.

That's my summary of the peer-reviewed research on the reliability of popular journalism.

It's similar to the literature on outcomes of surgery, which depends on the individual surgeon and the institution. Nbauman (talk) 18:10, 26 March 2009 (UTC)

Could you cite a couple of examples of this peer-reviewed research? I'm not sure exactly what is being talked about. For more on TimVickers's proposed wording, please see #Popular press accuracy below. Eubulides (talk) 18:47, 27 March 2009 (UTC)

(Undent) Nbauman, you've just gone circular on us. How do you tell if the newspaper story is accurate? By determining whether or not the journalist "writes an accurate story". That's not helpful. (Are you aware that this section already says that the coverage ranges "from excellent to irresponsible"? Perhaps you forgot that we already acknowledge the existence of excellent work -- while still recommending a solid scientific source for any scientific fact.) WhatamIdoing (talk) 17:56, 26 March 2009 (UTC)

I haven't gone circular. I was responding to your statement that it would be too difficult to "divine the reporter's level of (relevant) experience."

I was making a parenthetical comment that it's not necessary to divine the reporter's level of experience. If he's accurate, it doesn't matter how much experience he has. Some young journalists without much experience in journalism do very good work. (I disagree with Dentzer on that point.) Nbauman (talk) 18:23, 26 March 2009 (UTC)

Okay, let's take this from the top: The question we are addressing is "Is this popular press source sufficiently accurate that I can just use it as a source for a specific, concrete statement of scientific fact, without checking it against the proper scientific literature?"

• Dentzer says you might be safe trusting an experienced medical writer. You disagree with Dentzer (but complain that we misrepresent her).
• You say (without sources) that you might be safe trusting an accurate medical writer. Apparently you mean something more like "has a reputation for accuracy" instead of "wrote this specific story accurately" (which would be circular).
• I say this is all irrelevant, because the Wikipedia editor doesn't know whether the writer should be considered experienced or accurate.

I also note that you acknowledge above that the facts should be checked against the proper scientific literature -- "Wikipedia articles should be checked against the original published research or peer-reviewed review articles for accuracy" -- which is exactly what we have recommended all along.

So what's the problem, again? Do you want the editor to first figure out the reporter's reputation before double-checking the facts in the literature? What exactly is wrong with our existing recommendation that the editor skip that potentially time-consuming and error-prone step and just go straight to the literature, since (1) the editor will have to check the literature anyway and (2) we'll still rely on the literature if there's a conflict between the two sources, no matter what the reporter's reputation is? WhatamIdoing (talk) 19:30, 26 March 2009 (UTC)

On Wikipedia, people use the popular media (particularly the BBC for some reason) as the source for statements in articles. I think it's acceptable to use the BBC, New York Times, Wall Street Journal, or other sources, if it passes the checklist that Schwitzer and others have offered that I linked to above. You treat it like anything else under WP:RS.

Because this is Wikipedia, anybody who has the knowledge and access to the journals can check the news story against the journals. If the original news story checks out against the original source, you leave it in. If you think the news story misrepresents the original research, you can change it.

Why not go straight to the original journal article? For many reasons. First, most news stories are accessible to the public, while most peer-reviewed journals are not. Several million people read the New York Times. JAMA has a circulation of about 300,000. Second, it's easier for a Wikipedia editor to misunderstand a journal article than a news story by, say, Lawrence Altman in the New York Times. Third, news stories that meet Schwitzer's checklist will get comments by other experts in the field who may point out weaknesses in the journal article.

I don't know if you're an MD, but I interview a lot of academic MDs who publish in the NEJM, Lancet, JAMA, and similar journals -- and they themselves often get their information about clinical studies first from the New York Times. I was at a meeting recently trying to find a doctor who could comment on the JUPITER study in NEJM, but everybody there -- published academics -- kept telling me that they had only read the New York Times story.

You seem to believe in a romantic world where scholars get their information directly from the peer-reviewed journals, and eschew the lowly New York Times. But that's not the real world of medicine. Even leading doctors get a lot of their information about new studies from the New York Times. Nbauman (talk) 21:52, 26 March 2009 (UTC)

BTW, here's an example of a news story on Forbes that adds something clinically important that wasn't in the original BMJ article. The BMJ article reported that women who quit smoking before their 15th week reduce their risk of premature birth and small birth weight to normal. The Forbes story got a comment from another OB/GYN who pointed out some limits to the BMJ article's conclusions, such as the fact that the improved birth weight didn't translate into actual improvements in newborn health, and that it didn't tell whether pulmonary disease also returned to normal.

So a news story that meets Schwitzer's criteria can be more reliable than the original journal article, by better defining the limits to the study. Nbauman (talk) 17:17, 27 March 2009 (UTC)

• The above examples all seem to be highlighting primary studies, which WP:MEDRS properly warns against. People in many fields get their first reports about notable results from the newspaper, but there's a huge difference between being alerted to a new event, and getting an accurate, authoritative, and in-depth study. It's fine if my doctor hears about a new drug trial first from the New York Times, but I certainly wouldn't want my doctor prescribing it for me based on the NYT article!
• I think much of the current dispute comes from the difference between news articles and Wikipedia. Wikipedia is not a newspaper, and it rightly frowns upon following new stuff too closely (see WP:RECENTISM). In the vast majority of medical areas, Wikipedia articles should focus on what's known reliably and reproducibly, and should avoid reporting the very latest studies, regardless of whether it cites the studies directly or cites news articles about the studies. Since news articles focus heavily on the latest studies, most news articles should be avoided for that reason alone.

Eubulides (talk) 18:47, 27 March 2009 (UTC)

Eubulides, you're not a medical doctor, right? Nbauman (talk) 20:58, 27 March 2009 (UTC)

My personal qualifications are irrelevant to this discussion. Eubulides (talk) 21:57, 27 March 2009 (UTC)

I think you ought to talk to your doctor about where he gets his or her information from, and how much he or she uses the peer-reviewed medical journals to get his information.

You seem to think doctors read the major journals every week and that's where they get their information from. If you knew more about the real world, you'd know that those doctors are few and far between. There are studies of where doctors get their information, and the journals aren't that significant. Nbauman (talk) 23:11, 27 March 2009 (UTC)

And others read the synopsis provided by the likes of Uptodate which reference the original research. We should be trying to create a open source Uptodate and to do this we need to primarily reference the literature much like they do.--Doc James (talk · contribs · email) 23:26, 27 March 2009 (UTC)

There are several services like that. Another popular one is Journal Watch.

But a surprisingly large number of doctors get their medical news from the New York Times. (They look up the journal articles as necessary, and sometimes it isn't necessary.) Several reporters have a particular reputation for accuracy. Lawrence Altman is an MD, and he knows MDs read his work. Gina Kolata is a PhD. Is Uptodate any more reliable than Altman or Kolata? Nbauman (talk) 03:01, 28 March 2009 (UTC)

All too frequent

In my book, "reliable" does not include sources that are "all too frequently...wrong or misleading". Nbauman continues to argue that because newspapers can sometimes be accurate, we should regard them as reliable. In addition, he continues to avoid the "suitability" argument that even if they were reliable sources of medical information, they aren't suitable as sources for such in an encyclopaedia (see archive, for example, the "Bell curve" discussion). As DGG says below "No responsible encyclopedia could admit non medical sources to the discussion, except to discuss the ethical or political implication of the data. Lay sources testify to the lay understanding of science, not to science." I find this determination to "not get the point" after nearly two years of discussion on these talk pages to be quite extraordinary, verging on disruptive. Colin°^Talk 22:54, 25 March 2009 (UTC)

A lay source is a lay source, accurate or otherwise. Because it is open to accusations of inaccuracy, even if an sich reliable, it should not be used. It is as simple as that, and I agree with Colin that after so much time it should be clear that most voices here are in favour of a restrictive policy on lay sources. JFW | T@lk 23:44, 25 March 2009 (UTC)

Colin, the major medical journals, such as NEJM, JAMA, BMJ and Lancet, have been wrong or misleading in the past. For example, NEJM published articles promoting Vioxx, Lancet published an article claiming that autism is caused by vaccines, etc.

In your book, are the NEJM, JAMA, BMJ Lancet all too frequently wrong or misleading? Or are they acceptably not too frequently wrong or misleading? Nbauman (talk) 23:50, 25 March 2009 (UTC)

Oh for heaven's sake, do you ever see a newspaper or TV show publishing corrections? In both your examples, the journals made a point to publish large corrections. With the popular press you'd never know. Simply a ridiculous analogy. Also, when the medical journals mess up this is not due to inaccuracy but due to errors in judgement (Lancet) or misrepresentation by the original authors (NEJM). JFW | T@lk 23:56, 25 March 2009 (UTC)

I agree with Colin, the popular press needs to be used with care and never as the sole source for a fact or figure. The argument that because experienced and scrupulous journalists often get their stories right, so all news stories should be acceptable as a sole source in medical articles does not convince me. Tim Vickers (talk) 00:22, 26 March 2009 (UTC)

JFW, the point I'm making is that it's meaningless to say that newspapers are "too frequently" wrong or misleading. Any frequency of being wrong or misleading is "too frequently." Even prestigeous medical journals are wrong or misleading sometimes, and that's "too frequently" too.

As for corrections, yes, every reliable newspaper and TV show does publish corrections. (I've always made a distinction between reliable and unreliable news media.) I see them every day. The New York Times has a corrections page that runs every day [1], like the WSJ, and every major newspaper. The New York Times has a reader's editor, National Public Radio has an ombudsman.

If you're saying that the medical journals always make corrections while the newspapers don't, that's wrong. A pharmacist, Jennifer Hrachovec, was challenging the NEJM for at least 5 years on its Vioxx-VIGOR study, and Drazen refused to correct it. The WSJ, not the NEJM, published her complaints, and of course she turned out to be right. Nbauman (talk) 00:28, 26 March 2009 (UTC)

Tim, the only reason you're saying that is because the editors here are distorting my position (straw man). I have never said that the popular press should be the sole source for anything.

The argument that because experienced and scrupulous journalists often get their stories right, so all news stories should be acceptable as a sole source in medical articles does not convince me.

That doesn't convince me either. I never said it. I said that because experienced and scrupulous journalists usually get their stories right, we can use their news stories as one source for medical articles. But we shouldn't use irresponsible news stories. I'm simply following WP:RS.

Where did you get the idea that I am advocating this nonsense about using news stories as the sole source? Nbauman (talk) 00:38, 26 March 2009 (UTC)

Oh I see, reading over the meandering "discussions" above it looks like the topic now has little no relation to the text we are editing. It would be much more useful if you were to offer specific and concrete suggestions on how to improve the text of this policy. Tim Vickers (talk) 00:50, 26 March 2009 (UTC)

The problem is that the text we are editing misrepresents the sources that it cites in its support. I believe the text of the policy would be improved if it did not misrepresent the sources. I've given specific and concrete examples of how it misrepresents the sources, and how it can be improved. Unfortunately I'm outnumbered. That's Wikipedia for you. Nbauman (talk) 02:43, 26 March 2009 (UTC)

Any summary necessarily omits some of the summarized text, but that doesn't mean that it's misrepresentation. Your proposed changes would make the text a less-accurate summary of Dentzer, and would not be improvements. Improvements to the text are possible, but this discussion has beaten the topic into the ground enough that it's probably time to move on. Eubulides (talk) 03:13, 26 March 2009 (UTC)

Eubulides, Dentzer said in her introduction, "Often these messages are delivered effectively by [experienced] reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading."

I propose to leave all that in, or to summarize it, leaving in the parts about good and bad journalists. You think that would be less accurate.

You think it's more accurate to delete the first part about reporters performing effectively, which goes against your personal opinion, and leave in the second part about journalists being wrong or misleading, which supports your personal opinion.

I think that's amusing. Nbauman (talk) 22:01, 26 March 2009 (UTC)

Please see # Popular press accuracy below. Eubulides (talk) 18:47, 27 March 2009 (UTC)

The worst

I just now reverted a change of wording "the worst news articles too often convey wrong or misleading information about health care". This change is definitely not supported by the cited source, Dentzer 2009 (PMID 19118299), who says that wrong or misleading information is conveyed "all too frequently", and who does not confine her criticism to the "worst" articles (she does writes that lack of context in medical news is a "worsening" problem, but that's quite a different matter).

Regardless of our personal opinions on this topic, we do have an obligation to summarize our sources as accurately as possible within the space constraints here, and conjuring up a "worst" here does not fulfill that obligation.

For more on this topic please see # Popular press accuracy below. Eubulides (talk) 18:47, 27 March 2009 (UTC)

NPOV

I am tagging NPOV because this essay conflicts with our policies on reliable sources and neutrality. It was recently used in an attempt at removing a reliable source about a social issue [2]. From the start, this essay was an attempt by pov pushers with a conflict of interest to use an essay to further their pov pushing and silence the other side. Among the supporters, Jfdwolff, MastCell, Cyclonenim, Casliber, Mattisse and probably others admitted a conflict of interest. I was lied to when I was assured that this essay would not be used for removing information about social issues. --Mihai cartoaje (talk) 09:02, 10 March 2009 (UTC)

Mihai, many medical issues are social issues as well, but that doesn't alter their fundamental nature. Xasodfuih (talk) 11:26, 10 March 2009 (UTC)

The statement "there is evidence that the neuroleptic drugs themselves cause loss of brain tissue" is not a social issue but a medical statement that must be sourced per WP:MEDRS and WP:V. This is an interview in a newspaper, of a scientist discussing research they haven't yet published, and in which they are very careful not to state "cause" but just a correlation with increased exposure to "drugs". It only becomes a social issue (why are people being given harmful drugs) if the fundamental assumption (the drugs are harmful) has been adequately proven.

I am really puzzled why this is regarded as a neutrality issue. Nancy Coover Andreasen is just the sort of medical expert MEDRS would like to cite once she publishes her results in a respectable journal. She's not on "the other side". Colin°^Talk 12:18, 10 March 2009 (UTC)

To backtrack on the article history a little, I was for including this interview until I, with the help of others, discovered that four other studies in this area have already been published (two in humans, two in primates), and I've added them to the article, initially directly, later through reviews, once I've found which reviews mentioned them—I still find this aspect of WP:MEDRS unwieldy at times. Andreasen's interview doesn't give enough details on her findings; given that and the delicate nature of these studies, plus the inclusion of already published studies in the wiki article, I don't see why waiting for her latest work to get published is such a big deal; currently the article read "In first episode psychosis typical antipsychotics like haloperidol were associated with significant reductions in gray matter volume, whereas atypical antipsychotics like olanzapine were not.[93] Studies in non-human primates found gray and white matter reductions for both typical and atypical antipsychotics.[94]". Xasodfuih (talk) 12:43, 10 March 2009 (UTC)

Mihai cartoaje, as you rightly say newspapers per overall WP:RS can be used as in this example to confirm a social aspect, namely that a person holds a view (likewise if the President states he thinks the economy will be helped by an action) although I do not know enough about that person & the interview in question as to whether of sufficient "worth" to warrent inclusion or not in the article. However that is not the same as confirming that the view is correct or supported by data that the (real-world) majority can agree upon (i.e. interest rate or exchanges rate might change but that does not lessen that Obama had the view, nor conversely override the fact that the economy really might not been helped). Newspapers (in this regard) confirm the truth of a person making a statement, not the truth of the statement made. It is for that proof of fact (rather than social commentary of opinions being held) that WP:MEDRS helps clarify sources that may be helpful. Likewise any number of newspapers can confirm that Prince Charles holds great belief in homeopathy, but that is not taken as proof or otherwise that homeopathy actually works (for which need peer-reviewed scientifically undertaken controlled ...etc etc), his views on some modern building architecture being a monstrous carbuncle is highly notable being from who said it and the public debate this started - indeed I would tend to agree with him on this - yet buildings are not abscesses :-)

WP:MEDRS has been accepted as a guideline, it is not an essay. But note the top description "a generally accepted standard that editors should attempt to follow, though it is best treated with common sense and the occasional exception" - "generally" does not mean everyone, and I would agree even WP:RS should only seek to confirm the state of knowledge per WP:NPOV and not some abolute truth as we're not WP:SPOV. I fail to see a conflict therefore between the various WP help pages: none are written in stone (WP:IAR), all need sensible interpretation and on occassion discussion between editors as to how best to work on some specific aspect within an article :-) David Ruben ^Talk 13:20, 10 March 2009 (UTC)

(having edit conflicted with Xasodfuih) Yes Xasodfuih, I think one might use (not same as should include, for that is an editorial decission in writing a well rounded section) the interview to confirm her view, but it would need be clear then that a view of one person rather than an absolute fact (a subsequently submitted paper may be accepted, but could also be rejected by a journal). David Ruben ^Talk 13:25, 10 March 2009 (UTC)

It's like a controversy during the Olympics. For example, were the fireworks real or computer generated? The Chinese organisers have their view, the media have their view and bloggers have their view. A neutral article would cover all these views. --Mihai cartoaje (talk) 14:23, 22 March 2009 (UTC)

I resent Mihai's continued suggestion that I might have a conflict of interest. JFW | T@lk 16:39, 10 March 2009 (UTC)

I don't see anything in the above discussion that suggests that WP:MEDRS is in conflict with Wikipedia policy, that it is particularly relevant to social issues, or that it has been used to "removing a reliable source about a social issue". Surely this thread belongs on Talk:Schizophrenia, not here. Eubulides (talk) 16:46, 10 March 2009 (UTC)

I'm concerned about Mihai's 'story' here. The discussion on the article's talk page is reasonable and making progress. MEDRS was cited as one example of a policy that says the amount of attention stuff gets in Wikipedia is based on the amount of attention they get in print. It's a bit odd to turn that widely accepted principle into "removing a reliable source about a social issue".

It's also a bit odd to turn "volume of gray matter" into a "social issue". Mihai, please don't confuse "the reason I believe this" with "what I believe". A reduction in gray matter volume (or its absence) is a concrete, measurable, scientific fact. The conclusions that someone draws from this fact (for example, that there should be government policy changes) are social issues.

As for the specific topic: actual causation may be hard to prove in humans. You'd have to prove that changes in schizophrenic patients were due to drugs and not to disease processes. (I understand that just the opposite has been proven in bipolar patients: manic episodes kill brain cells.) WhatamIdoing (talk) 17:11, 10 March 2009 (UTC)

As I commented on the talk page, the question of efficacy or safety is not in any sense asocial issue. No responsible encyclopedia could admit non medical sources to the discussion, except to discuss the ethical or political implication of the data. Lay sources testify to the lay understanding of science, not to science. DGG (talk) 02:28, 11 March 2009 (UTC)

It's almost becoming a bit hard to take Mihai seriously - but then I am a "POV-pusher with an admitted conflict of interest", just like everyone else who disagrees with him. MastCell ^Talk 05:03, 11 March 2009 (UTC)

Applied kinesiology: confused use of research sources

In this section of the article, there is a confusing use of sources. This situation has occurred because of the edits made by science-based editors on one side, and a board member of the ICAK on the other side. He has attempted to present this pseudoscience as a scientifically legitimate discipline. The best research has consistenly shown it to be hogwash. Please take a look and you'll be rewarded with a fascinating glimpse into the 10th most used chiropractic technique. -- Fyslee (talk) 05:30, 17 March 2009 (UTC)

Mediation

A user has requested mediation on this issue. Jmh649 is here to help resolve your dispute. The case page for this mediation is located here.

I have not been involved and have added some comments similar to those above.--Doc James (talk · contribs · email) 21:51, 24 March 2009 (UTC)

Popular press accuracy

(This follows up on the #Medicine and the popular press thread above, which was getting a bit long.)

Above, TimVickers suggested inserting this text somewhere:

Although some articles in the popular press are accurate, many others are inadequate, misleading, or even completely wrong.

I have a few ideas for making the proposal more concrete and better-supported by the cited source, Dentzer 2009 (PMID 19118299). I just now reread the above thread and Dentzer, and with all this in mind, I suggest that in WP:MEDRS #Popular press, we make the following change (deletions struck out, insertions italicized, and changing the preceding comma to a period):

and Although medical news articles often deliver public health messages effectively, they too often convey wrong or misleading information about health care, partly when reporters do not know or cannot convey the results of clinical studies, and partly when they fail to supply reasonable context.[PMID 19118299]

This captures Dentzer's thrust more accurately (albeit at more length), and mentions a positive thing she has to say about the news, which seems to be the main point of contention here. Eubulides (talk) 18:47, 27 March 2009 (UTC)

I'm concerned we are placing too much emphasis on trying to work out and reproduce the opinions and words of one author in one paper, which is an opinion piece, not a product of research. The article is not just concerned with accurate reporting (though she does give examples of misleading or incorrect reporting) but whether the underlying health message is being conveyed successfully. The wrong message may be conveyed even if the reporting of the "new" story is accurate, because it is incompletely described, or is not placed in context with other research or health concerns. The statement that "Often these [public health] messages are delivered effectively" is actually the least convincing opinion in the article and she cites no research to back it up. Perhaps nobody questioned that the word "effectively" means "to have an effect". Public health messages are largely ineffective, and I'm sure there's lots of research on the subject. Indeed, newspapers are guilty of effectively spreading anti-health messages such as those from the anti-vaccination lobby, or over-simplistic messages such as the antioxidant theory encouraging the consumption of red wine, or waste-of-time-and-money messages such as the high coverage of many CAM "therapies". We must also remember that "medical news articles" with the highest readership are found in glossy women's magazines, tabloids, and television news. The author of that paper, along with the folk on this talk page, probably read a better quality of news than average. Colin°^Talk 20:06, 27 March 2009 (UTC)

Colin, I agree that we are placing too much emphasis on the Dentzer piece which you correctly point out is an opnion piece, not supported by data or research. If you want peer-reviewed research supported by data, you can follow the links on Schwitzer's web site. Nbauman (talk) 20:52, 27 March 2009 (UTC)

Eubulides, I read an article in the BMJ about doctors who murder patients, which cited several examples.

Would you agree that doctors all too frequently murder patients? Nbauman (talk) 20:54, 27 March 2009 (UTC)

I don't know about Eubulides, but I would not agree with the statement "doctors all too frequently murder patients". What a ridiculous example. Nbauman, you are seriously in danger of self-highlighting the sort of misunderstanding and misreporting of statistics that we are criticising journalists for. It is a cliché that "one avoidable death is a death too many", but fortunately the world doesn't get paralysed while we wrap it in cotton wool. Colin°^Talk 21:34, 27 March 2009 (UTC)

A recent change gave reporter inexperience as the reason for poor medical stories, claiming the cited article excluded "seasoned reporters". It does nothing of the sort. That is a logical error like saying that "weekdays are dry" based on a source that says "often weekends are ruined by rain". Colin°^Talk 21:18, 27 March 2009 (UTC)

Nbauman - stop edit warring. You were bold. It got reverted. Now discuss. Your reinsertion of text which is neither supported by the cited article nor supported by consensus of WP editors is extremely bad form. I strongly encourage you to have the decently to self-revert and engage in consensus seeking here before making contentious changes. Colin°^Talk 21:37, 27 March 2009 (UTC) WP:RS does not apply outside of article space, and you yourself regard it merely as an "opnion piece, not supported by data or research". Colin°^Talk 21:39, 27 March 2009 (UTC)

Colin, did you read Dentzer's article before making that change? She says:

Often these messages are delivered effectively by seasoned [experienced] reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health news by many other journalists is wrong or misleading.

We agreed that "seasoned" means "experienced."

So Dentzer said that "other journalists" are wrong or misleading, not experienced journalists. Nbauman (talk) 21:44, 27 March 2009 (UTC)

Four statements: [1] There exist seasoned reporters who have delivered messages effectively. [2] There exist other journalists who have conveyed wrong or misleading information. [3] Wrong and misleading information is conveyed by inexperienced journalists. [4] Wrong and misleading information is not conveyed by experienced journalists. 1+2 implies neither 3 nor 4. Please go read up on logic before suggesting I didn't even read the article. Colin°^Talk 23:24, 27 March 2009 (UTC)

Colin, please go back and re-read the word "other." Nbauman (talk) 03:03, 28 March 2009 (UTC)

"Other" means "those journalists not in the first set". The first set is defined as "seasoned reporters who have delivered messages effectively". The first set is not defined as "all seasoned reporters". For that, the author would have had to write "Seasoned reporters deliver health messages effectively." You are seeing (because you want to) an implication that strictly isn't there. However, I suspect Dentzer didn't worry about whether her statement would be logically analysed or mined for implications, because as I pointed out earlier, her use of the word "effectively" is careless. Colin°^Talk 10:44, 28 March 2009 (UTC)

I strongly disagree with the recent change "news articles too often can convey wrong or misleading information about health care particularly when they are written by by inexperienced reporters". Not only is it ungrammatical, it focuses on what is a secondary topic of Dentzer's article (lack of training) and misses her main point (lack of context in news articles). Also, it removes a "too often" that is an important point in Dentzer, and should be a point here. I agree with both Colin and Nbauman that this thread is focusing too much on Dentzer and too little on what the guideline should say. I frankly am dismayed by Nbauman's debate tactics on this talk page; they are not conducive to collaboration. Let's work out a change on the talk page first, and not edit-war the project page. Eubulides (talk) 21:57, 27 March 2009 (UTC)

Perhaps more importantly, this is not a mainspace article on Role of popular press in health care. This page is advice from Wikipedia editors to other Wikipedia editors. We don't need sources to "prove" that our advice is to use high-quality secondary scientific sources for statements of scientific fact, and we don't need to split hairs over the ideally and comprehensively neutral way to represent one person's published opinions.

There is no consensus for change. Nbauman remains offended that we're not showing enough respect to his favorite colleagues, and will probably always remain offended by our supposed scientism. None of the other involved editors want to lower our standard to accommodate his POV. I think we need to just stop talking at this point. WhatamIdoing (talk) 22:56, 27 March 2009 (UTC)

Agree completely. Scientific journals are the best source. This type of question can be decided by a vote if needed.--Doc James (talk · contribs · email) 22:59, 27 March 2009 (UTC)

Vitamin E

• An interesting example of a recent popular press distortion comes in the form of NYTimes recent coverage of vitamin E. For example, they painted a misleading picture of the aftermath of the SELECT trial [3] (which our selenium article reflects fixed). Much better commentary appears in Gann's accompanying editorial on the SELECT trial [4] or in the article's discussion itself [5]. The most striking difference is that while earlier trials were focused on selenium-deficient people, the SELECT men were high in selenium. Selenium is toxic at higher doses. Big surprise. Similarly, NYTimes recently highlighted the "increased mortality" from vitamin E in an article titled "Extra Vitamin E: No Benefit, Maybe Harm". As I pointed out in my my comment, they hardly emphasize that the increased mortality occurred only at 400+ IU (27 times the RDA) and excluded the fact that at a meta-analysis found a non-statistically significant reduction in deaths at 100 IU. They also excluded the fact that meta-analysis was hesitant to generalize this to the non-elderly population. What they also don't mention is that, according to the latest NHANES, 90% of Americans don't get the RDA of 15 IU. From that perspective, basic vitamin E supplementation looks prudent, and indeed the ATBC trial found benefits for those with higher serum levels. Despite the issues with NYTimes coverage, it was added to the orthomolecular medicine article by experienced medicine editor MastCell [6]. Although I pointed out the problems in E's coverage at that article's talk page, no one seems interested in providing a neutral overview of poor E. II | (t - c) 23:35, 27 March 2009 (UTC)
  • I'd rather not discuss specific content issues here, especially when they're as historically intractable as vitamin supplementation. From the perspective of WP:MEDRS, you're actually dismissing the New York Times article far too hastily. In fact, it does note that the excess mortality was seen with high doses. An entire subsection of the Times article is entitled "Too Much of a Good Thing", and states:

    Recent studies have even suggested that at the high doses many people consume, vitamin E could be hazardous. In November 2004, the American Heart Association warned that while the small amounts of vitamin E found in multivitamins and foods were not harmful, taking 400 International Units a day or more could increase the risk of death. ([7])

    That's followed by a fairly detailed breakdown of recent studies on the subject. So I think the Times piece is actually an example of responsible mainstream-media reporting, which is one reason I cited it. Another reason is that the citation occurred in orthomolecular medicine, which specifically involves very large doses of vitamins and where the potential harms of such megadoses are relevant.

    I can't fault the Times for "excluding" a non-significant finding - I'd be more inclined to praise them for doing so. And the Times piece reaches the same conclusion as Gann's (excellent) editorial: Vitamin E pills aren't a magic bullet. MastCell ^Talk 00:40, 28 March 2009 (UTC)

• Dentzer's point is that "when journalists ignore complexities or fail to provide context, the public health messages they convey are inevitably inadequate or distorted". That's exactly what happened there, when the average vitamin E intake is around 7 mg/day [8]; the ATBC study (not covered) found benefits for moderate supplementation, and the meta-analysis found that "low-dose" vitamin E (still several times above the RDA) reduced deaths; the probability that it was due to chance was 20% -- not stat. significant, but not worth discarding either. Based on the literature, the article could also have reasonably been titled: "Extra vitamin E: Good in moderation". It was quite easy to come away from that article thinking that vitamin E is scary and dangerous. Further, the results of the meta-analysis were challenged [9]; anyway, to get back on topic, I actually think Dentzer is being somewhat selectively cited, but the section is fine overall. One thing to keep in mind is that papers can sometimes be extremely technical; in those cases a lay summary is necessary. As far as OMM, I'm disappointed you didn't notice the major problems in the vitamin E section (not the least the statement that "initial hopes for OMM were based on observational studies" -- implying that mainstream, rather than OMM fringe, was responsible for the RCTs), and hopefully you'll comment the next time I bring it up. By the way, looking at Gann again, I'm not so impressed by the way he attributes the results of earlier studies to chance when they were clearly different populations. II | (t - c) 07:10, 28 March 2009 (UTC)

• Agree with MastCell that this thread on Vitamin E doesn't belong here; presumably it belongs in Talk:Orthomolecular medicine. Eubulides (talk) 16:38, 28 March 2009 (UTC)

Scientific evidence for guideline

WhatamIdoing, you've got it exactly backwards. I have always been a strong critic of medical journalism, and I agree with most of Schwitzer's work because Schwitzer is based on evidence. Much medical reporting is terrible. I don't know if "most" of it is terrible, but I'll go with the evidence.

I'm flattered when I see articles like Dentzer's, because I'm one of those experienced journalists she talks about. I write for doctors, my work is edited and reviewed by doctors, and if I didn't get the story right in every respect, my work would be tossed out pretty fast.

My difference with you and the other editors here is that I believe that evaluations and criticisms of medical journalism should be based on the same kind of scientific evidence that we use in medicine and elsewhere in science. That's what I'm offended at -- your abuse of science and contempt for scientific method. You're not using high-quality secondary scientific sources at all.

You're making wild claims about journalism based not on scientific evidence, like Schwitzer's work, but on your own personal prejudices. I won't speculate on the psychological roots of those prejudices, but you can't support your broad overstatements with facts.

You cherry-pick articles like Dentzer's to find excerpts that you think support your position, and quote them selectively and misleadingly, even though you misunderstand the whole essay.

I've seen something similar, where alternative medicine advocates will pick out excerpts of medical articles to support their argument that all doctors and all drug companies are harming people.

Your basic fallacy is to condemn all journalism in the popular press, when the scientific data displays a much more interesting pattern: some of the popular press is accurate and some is not. So you're really missing the point.

It's paricularly ironic that you claim to be supporting "high-quality secondary scientific sources", when you're doing exactly the opposite. Except for Schwitzer, your entire guideline is based on personal opinion and personal prejudice and ignores those sources. Nbauman (talk) 23:45, 27 March 2009 (UTC)

• I dunno, are any of the other Wikipedia guidelines or policies rigorously supported by scientific evidence? Although the scientific method is quite useful for resolving scientific disputes, one can't use it for everything.
• Hmm, come to think of it, shouldn't any effort to document (and perhaps even scientifically support) medical journalism be put into mainspace? I just now created the stub Medical journalism to do that. I get the feeling that if even 10% of the effort expended on this thread had been put into Medical journalism we might have a useful article there by now. (And no, that's not a scientific estimate....)

Eubulides (talk) 16:38, 28 March 2009 (UTC)

A secondary source dilema

Here is a dilema that I have noticed particularly with books. Whilst it is for obvious reasons preferable to use secondary sources wherever possible, is this always the case? One scenario I saw was when reading a high quality secondary source in a book which is widely used especially by medical students where it stated that benzodiazepines are GABA reuptake inhibitors or increase GABA levels for their therapeutic effects which is clearly false as they work at benzodiazepine receptors enhancing the effect of GABA at the GABAa receptor via benzodiazepine receptors. This scenario didn't happen on wikipedia but using it as an example of inaccurate info in books, but something similar happened in the past few days where on the surface seemingly high quality books on psychiatry and psychopharmacology were cited but on closer inspection the authors were employed by literally over 20 pharmaceutical companies and they had cherry picked 2 or 3 primary sources and then distorted the results of the primary sources and reinterpreted them and falsely represented the primary sources without any evidence but their opinion. So what would happen in that scenario? To be clearer though the sections of the books were not doing an extensive review of the literature but were basically touching a little on every aspect of psychiatry and psychopharmacology. So that brings up another question what happens if the secondary source is not doing a meta-analsysis or extensive review of the literature but are simply touching on one aspect of a condition or drug or syndrome etc? Also is a peer reviewed meta-analysis or review superior to a non-peer reviewed book? Is a peer reviewed secondary source published by people with ties to a drug company or some other conflict of interest equal to an independent peer reviewed secondary source? I guess my point and dilema is this. The use of secondary sources are viewed higher on wikipedia because they are seen as being an independent outside opinion but what if the author(s) are not independent and/or they can be shown to be distorting data? My main concern is secondary sources which are not peer reviewed as this area seems to be most open to abuse by those with a conflict of interest. Currently the recommendations is books which are not necessarily peer reviewed and open to abuse, opinion, synthesis, reinterpreting results, distorting facts are declared as "excellent" (secondary) sources. I disagree and think that the problems of books needs to be addressed. There are some good medical books which are high quality don't get me wrong but you see the dilema that I see and problem for misinformation getting inserted into articles. I think peer reviewed secondary sources trump non peer reviewed secondary sources and that many (but not all) books in general are not excellent sources of reliable citeable data. Infact I think primary sources are quite often superior to many books depending on the subject matter being quoted for reasons that I have just explained.--Literaturegeek | T@1k? 23:32, 1 April 2009 (UTC)

I have striked out some of my text as on closer inspection I see that you differentiate between non-reliable medical books and reliable trusted publications, but still have some concerns mentioned above.--Literaturegeek | T@1k? 15:54, 2 April 2009 (UTC)

That's kind of a long question, and it sounds like this is really more of a question about a particular case, but in general the answer (I think) is that peer-reviewed sources are generally (although not always) preferred. Conflict of interest is of course an important issue, but in practice almost everybody has a conflict of interest of one form or another, and a review should not be disqualified simply because a reviewer has a conflict of interest. Eubulides (talk) 16:45, 2 April 2009 (UTC)

[Edit conflict with Colin, posting at same time] Sorry for length of questions. This was the book that I was talking about. Would this book be considered superior to an FDA review or Committee on Safety of Medicines review? This book is cited in an article and I have no intention of deleting it so I am not in a conflict with anyone per se. It was just when reading the book and comparing it to their sources I noted how they had distorted or even falsified the conclusions of the sources they were citing. Also they appeared to be giving best practice guidelines based on taking one or two primary sources out of context and also synthesising without any evidence in conflict. I guess I noticed a potential occasional serious problem with using books over more reliable peer reviewed secondary sources. Then of course the above example I mentioned where a trusted book source got the mechanism of benzodiazepines completely wrong is another example. I am not against using books and more often than not they do make good sources to quote and infact I used one today myself. I guess I am asking does a secondary peer reviewed meta-analysis or review source trump this example of a book? Or does FDA review or CSM review or department of health guidelines trump it?--Literaturegeek | T@1k? 17:56, 2 April 2009 (UTC)

Don't overplay the "independent" aspect of secondary sources. The aspects of secondary sources we like are that they are written by experts and they have undergone some form of expert review (peer or editorial). When wikipedians choose primary sources we end up with the selection & analysis of those primary sources done by nobodies, with no reputations to uphold, and (often) no serious review. So I find your concern about bias in the secondary-source authors to be a red-herring -- there is bias in the primary sources and there most certainly is often bias among wikipedians (which is undeclared, whereas conflicts-of-interest are declared in some sources). A medical book published by a serious publisher will have undergone some form of review. If you have found mistakes in a student textbook, perhaps you should contact the publishers. Colin°^Talk 17:51, 2 April 2009 (UTC)

Colin, I think perhaps you misinterpreted me or perhaps it is my fault in how I worded the issue. I was not in anyway shape or form saying that primary sources are superior to secondary sources. I was raising an occasional isolated problem of what happens basically if a secondary source says something which is nonsense. For example should we change the mechanism of action of benzos to say they are reuptake inhibitors because a good quality book made a mistake? Or in such a scenario would a peer reviewed secondary source trump it so we can add accurate data? What happens if the book is demonstratly misrepresenting their sources etc. I was talking about isolated examples. These examples didn't happen in real life on wikipedia but I just noticed a potential from spotting several inaccuracies and thought that I would raise it. I have noticed this is less problematic in the peer reviewed literature especially secondary sources which have been published in medical journals. The inaccuracies I have spotted in sources (particularly books which often briefly skim a topic) are not necessarily POV issues but just a matter of an inaccurate factoids. Another one I have noticed is flunitrazepam is short acting when it is long acting.--Literaturegeek | T@1k? 18:03, 2 April 2009 (UTC)

• One of the reasons that User:Paul gene strongly opposed the emphasis on reviews was that, in his words (if I recall correctly), they are often "written by pharmaceutical hacks". These days the tide has begun to turn and, if you look hard enough, you can find unbiased information, as evidenced by the recent Cochrane review on St. John's wort (PMID 18843608) and the recent review on the NEJM on publication bias in antidepressants [10]. Sources without conflicts of interest should be preferred in my mind, or at the least included, and we should probably have a section on it in this guideline. II | (t - c) 18:41, 2 April 2009 (UTC)

This guideline covers the whole of medicine. Drugs and the pro/anti-pharma controversy is really a small part. Folk get worked up about conflicts of interest and bias when much of what we should be writing about on WP, and pushing towards FA, is bread-and-butter non-controversial medicine. We must always bear this in mind when giving and suggesting changes to guidance on sources. Much of research is done and reviewed by honest folk earnestly trying to work out why we get ill and how to fix it. It is just a little wearying to keep hearing that secondary sources are full of bias, written by hacks in the pay of 20 pharmaceutical companies, and that if you'll only let me choose, weigh and interpret the primary sources instead, then I'd do a better job... Colin°^Talk 19:10, 2 April 2009 (UTC)

Speaking of misrepresenting primary sources, you appear again to be misrepresented me. I never said nor implied that I should resynthesise an authors opinion. I was asking what do we do in that scenario? Do we accept the demonstrately intentional or unintentional inaccurate factoids or opinions of a book which often briefly touched on one aspect of medicine,,, or does a peer reviewed secondary source such as a meta-anaylis or review or FDA, department of health etc etc trump it? I came here just to ask for guidance on what to do. Initially I misread the book guidelines on this project and striked out my text above but still wanted some guidance or discussion on what to do in these isolated scenarios I have mentioned. Instead I appear to be getting responses trivialising and mocking me as an idiot by distorting what I was saying and asking.--Literaturegeek | T@1k? 19:28, 2 April 2009 (UTC)

Secondary sources are usually more reliable than primary sources. The fact that a handful are not just means that you should, in these cases, use several secondary sources to give a broad view of the range of opinions on the matter. What is not appropriate is for us to use primary sources to cast doubt on the conclusions of secondary sources - that is original research. Tim Vickers (talk) 19:15, 2 April 2009 (UTC)

Thank you Tim this is what I was asking for, some guidance. Could you also give me your opinion on whether official national reviews by say the FDA or Department of Health, meta-analyis or review in a journal would be superior to a book? What happens if a book says for example SSRIs work by stimulating the release of serotonin when the truth is they block its reuptake or something like that? Do we delete the book ref and replace it with a more accurate secondary source? Or do we leave the mistake of the book in even though we know that it is false? I guess I would like this project to state whether reviews or meta-analysises (secondary sources) in medical journals are superior to a (secondary source) book if content is in dispute.--Literaturegeek | T@1k? 19:28, 2 April 2009 (UTC)

That's really a question for the RS noticeboard, this page is for discussing this policy, not specific sources or specific disputes. However, the advice given already at Wikipedia:Reliable_sources_(medicine-related_articles)#Choosing_sources seems to apply well to this instance. Tim Vickers (talk) 19:33, 2 April 2009 (UTC)

Okie dokie Tim, problems with specific sources, try and sort out on article talk page or else if that fails go to RS noticeboard is what you are saying I believe. So no need to alter guidance. You have answered my question, thank you.--Literaturegeek | T@1k? 19:47, 2 April 2009 (UTC)

Any source that is rather general and broad (such as a student textbook) is likely to be inferior to a more specific book or paper. It may well have attempted to condense a complex subject with a simple explanation that is in fact wrong. This may be the case for your benzo textbook. A specialist book aimed at medical professionals would be a better source, as would a relevant secondary article such as those you mention. Where you have several sources in disagreement then it may be relevant to note the disagreement among professionals within the article (as Tim suggests) or you could post a query on the article talk page or WP:MED talk page and ask for the opinions of others. You may find someone has access to a better definitive source. Colin°^Talk 19:51, 2 April 2009 (UTC)

Ok thank you Colin for your reply and good advice, makes sense.--Literaturegeek | T@1k? 20:00, 2 April 2009 (UTC)

Just to clarify it wasn't a "benzo book", but a clinical psychopharmacology text book (see above) and also another medical text book. Anyway like I said the inaccurate data wasn't cited on wiki so I was raising a potential issue I had noted with isolated books saying inaccurate things eg benzos are reuptake inhibitors or distorting sources. Anyway like Tim said discuss on article talk page to reach consensus, use better secondary sources or as a last resort go to RS noticeboard in such a scenario.--Literaturegeek | T@1k? 21:52, 2 April 2009 (UTC)

This has been bugging me and I would like to end this by saying that I apologise for the confusion generated by myself initially misreading (due to skim reading when half asleep) the policy regarding books and I now know why Colin reacted the way he did. He is only trying to defend wikipedia. Ironically I was trying to do the same in my posting and had nothng to gain and the policy actually reads the way I thought it should read but I misread it initially. Anyway I got my questions answered in the end so all is good. :) I would like to just end this conversation here by saying I support Colin's efforts in defending this project and wikipedia and I am glad we now understand each other's misinterpretation of each other. No need for anyone to reply.--Literaturegeek | T@1k? 02:15, 6 April 2009 (UTC)

Dubious section on green tea and HIV

Please take a look here. This is potentially deadly advice. -- BullRangifer (talk) 03:45, 14 April 2009 (UTC)