Wikipedia talk:Identifying reliable sources (medicine)

Re a recent change to wording / meaning

A change of wording (by NuclearWarfare) here that currently appears in the Biomedical journals subsection seems to me to be significant. The change is from

Research papers that describe original experiments are primary sources; however, they normally contain previous-work sections that are secondary sources (these sections are often incomplete

to

Research papers that describe original experiments are primary sources; however, they normally contain sections that place their research in the context of previous work. These sections may be considered secondary sources for the purpose of this guideline, but note that these sections are often incomplete

The current wording seems to me to imply that such content may be considered an acceptable secondary source under certain circumstances. I feel the wording and positioning of this statement needs some attention to avoid unintended interpretations. So I made this edit for the sake of clarity, which hasn't actually survived (just for the record, backstory details here). Either way, such changes to the wording seem to me to have some substantial implications, and probably require consensus (or otherwise) to be reached here.

Fwiw, I don't have strong feelings on this, though I too would tend to support a flexible approach in such situations (I think). 86.128.169.211 (talk) 19:12, 29 June 2014 (UTC)

NuclearWarfare is correct: previous-work sections (that exceed a couple of sentences) are (usually) secondary sources. (The presence of analytical statements is key.)

The thorny question is really whether we want to advertise this fact to editors. Advertising it to editors like NuclearWarfare would be an excellent idea. NW's groks due weight and is already attuned to the risks of using these sections. Advertising it to a very inexperienced editor or a POV pusher would probably not be such a good idea. We may end up with refspam, creeping reliance on primaries, and cherry picking. I don't know what to recommend here. WhatamIdoing (talk) 19:53, 29 June 2014 (UTC)

Per WAID's considerations, I had two main thoughts about how we might handle the actual wording:

• that it's probably better to discuss use of research papers after (rather than before) talking about reviews;
• it's perhaps a good idea to specify rather clearly the sort of situation in which it's appropriate to source information contained in a research paper.

In this version (diff here) I felt I was probably interpreting both NW's thoughts and broader consensus (though I accept I should probably have discussed here before making the actual change). 86.128.169.211 (talk) 20:27, 29 June 2014 (UTC)

WhatamIdoing, maybe one thought would be to add in a stipulation that in the cases of contentious topics, primary sources are a no go when even citing the introduction as a form of secondary literature? That way we could allow primary literature introductions in cases where it's something that should be cited from a journal, but isn't anything likely to be in an area where a POV pusher could really do damage. If I'm citing a paper that lists in its introduction what pathogens penicillin has been used against, that would seem like a non-issue, but cherry picking a fringe paper that mentions a link to autism with vaccinations in its introduction would definitely be a cause for concern. So basically say primary sources can be used with caution in this sense, but if anyone raises a fuss, it's gone.

This is just a potential solution, but determining what would be a contentious topic would be subjective, so I'm not sure how feasible the idea would be. This could allow some flexibility where we can error on the side of caution, and just say that if someone at all disputes the addition of a primary source's content, it's then contentious and shouldn't be used. If we’re going to have some ambiguity in the guideline, I’d at least say have that wiggle room skewed towards the use of secondary sources rather than make it easy to put in primary sources. Personally I’d prefer just secondary sources for the reasons you mentioned above, but maybe this approach might work? Kingofaces43 (talk) 18:28, 14 July 2014 (UTC)

That's an interesting way to address the problem. I think it would work, at least if people are challenging it for actual/specific reasons, rather than the sort of "I challenge every sentence followed by a primary source from here to infinity" thing that we see every now and again. It's not very different in spirit from the line that we use at WP:ELBURDEN, which is that any external link that's is disputed should be removed: assume that it's fine, remove if there are indications that it's not. However, it's a bit more complicated, given that we're supporting article information. A weak source is sometimes better than no source.

NuclearWarfare, what do you think? Do you think that there's a risk of this being gamed? I'm a little bit worried about people adding REFSPAM, and moderately worried about people fussing that the source isn't good enough, but not being willing to replace it with a better one themselves. WhatamIdoing (talk) 22:44, 14 July 2014 (UTC)

Grok, now there's an interesting phrase that I haven't heard of before. I like it.

Anyway, as to the point raised by Kingofaces43 (had a hard time not reading that as "King of faces," which weirded me out a little): it's a defensible position. And I know how easy it is for "citogenisized" mistakes to spread even just within in the professional literature, because peer reviewers and editors often don't seriously review the background sections. But I think it's taken a little bit far. I just added some content to the [arthroplasty]] article last week that summarized the latest state of the meniscectomy literature. For that, I largely relied on the background sections of the major RCTs that have been published in the area (all in either the major general medical journals or the major speciality journals, so high quality sources). This information simply hasn't been well synthesized in reviews yet. It's a little too new (~seven years since the original trials started being published) and a little too controversial (partly because the observed data has high variance) for treatment guidelines. But I think it is still important for Wikipedia to reflect the consensus in the broader field. This change would take it a little too far in prohibiting that, at least as I read it. How about something like the following: Research papers that describe original experiments are primary sources; however, they normally contain sections that place their research in the context of previous work. These sections may be used in Wikipedia with care. As they are often incomplete, such citations challenged in good-faith by another editor should be replaced by a more comprehensive source [if possible?]. I'd welcome input. NW (Talk) 01:42, 15 July 2014 (UTC)

I would amend it to Research papers that describe original experiments are primary sources; however, they normally contain sections that place their research in the context of previous work. These sections may be used in Wikipedia with care. As they are often incomplete, such citations challenged in good-faith by another editor should be removed or preferably replaced by a more comprehensive source if possible. Verifiability and reliability of content are what makes up core policy for Wikipedia's content standards, so I'd rather see it be harder to add information from primary sources (although still possible). The purpose of this set up is to raise the bar of reliability for any content or topic considered contentious (which seems like it should be the case anyways), and to allow the possibility of primary sources in less contentious topics where there likely wouldn’t be a dispute anyways.

What I’m not sure about though is how this would affect the quality of articles depending on how active editors are. In other words, would it be easier for poor quality content and sources to creep in for lesser known topics, or is this happening already since they aren’t checked as often anyways? This approach would basically endorse that we accept lower quality evidence for some articles, but require more for others. I'm a little split on that thought. I think this is good in contentious topics so that we can stick to high quality sources and avoid a lot of the issues that come from POV, reliability, and other common problems in those discussions, but I’m not so sure about essentially endorsing what seems like a lesser standard for potentially less checked topics. Maybe it’s not really that different than what typically goes on now though? Kingofaces43 (talk) 15:08, 15 July 2014 (UTC)

@Kingofaces43: You're right that it's always best to use the highest quality sources possible, even for lower profile articles, but part of the consequences of certain topics being lower profile is that the highest quality sources don't always exist. I think that your phrasing is fine. Perhaps we can do more to emphasize that we really do mean that the challenge must be in good faith as to the editor's concern about The Truth^(TM) factualness of the statement. NW (Talk) 21:30, 19 July 2014 (UTC)

If you two wanted to take a baby step towards this, rather than plunging forward (which I'm not opposed to), then you could limit its application to "subjects, such as rare diseases, for which review articles and reference works are not abundant". Alternatively, you could try it out for a while and see what happens. We can always remove this statement if it proves to cause problems.

In response to a comment above, there are types of basic information that people expect an article to contain (e.g., basic information about signs and symptoms, epidemiology, diagnostic process, typical treatment), regardless of whether that information happens to be present in a review or reference book that was published in the last five or ten years. WhatamIdoing (talk) 23:48, 19 July 2014 (UTC)

Sorry I'm a little late in responding WhatamIdoing, but in a case of basic information like you describe, wouldn't that then be something unlikely to be contested? The type of content you're describing sounds like it wouldn't be contentious at all, and if someone did raise a fuss about the sourced content (then showing there is some contention) then the editors would have to go to more reliable secondary sources as they probably would do anyways to show the information is indeed factual and not some artifact from being in the primary literature. It seems like a fluid way to handle the situation, so I'm curious where you see an issue in your example? I'm not seeing one, but if you could explain what you're thinking of a bit more maybe we can figure out a way to tighten up the language a bit more. Kingofaces43 (talk) 16:32, 30 July 2014 (UTC)

Even if the specific information is unlikely to be challenged, good editors are usually dissatisfied with their work if there are zero sources in an entire section. WhatamIdoing (talk) 16:29, 11 August 2014 (UTC)

Avoiding Discussion of Causation with Evasion

Hello, Identifying reliable sources (medicine). You have new messages at Talk:Selective_serotonin_reuptake_inhibitor#Avoiding_Discussion_of_Causation_with_Evasion.
You can remove this notice at any time by removing the {{Talkback}} or {{Tb}} template.

Cannabinoid hyperemesis syndrome

It appears that Cannabinoid hyperemesis syndrome may be a medical article based on primary sources, contrary to the advice on this project page. But as a layman, I would like to hear from others better qualified to offer an opinion. Msnicki (talk) 22:51, 27 July 2014 (UTC)

Why Not Primary Sources?

Primary Sources are sources in which the authors directly participated in the research or documented their personal experiences. They examined the patients, injected the rats, filled the test tubes, or at least supervised those who did. Many, but not all, papers published in medical journals are primary sources for facts about the research and discoveries made. That being the case, why shouldn't Primary Sources be referenced in articles about medical subject matter? Ormr2014 (talk) 17:35, 4 August 2014 (UTC)

This issue has been discussed a ton on this page - just search this page and the archive for "primary" and you will find tons of explanations. You can also see an essay I wrote called "Why MEDRS"? here. The very short answer is: 1) all WP content should be sourced from secondary sources as per WP:RS. Secondary sources tell us how much WP:WEIGHT to give information - it is beyond your authority or mine as editors here, to make those decisions. 2) More specific to the biomedical space.... Scientists publish their work to talk to each other; we happen to live in an era where we can witness that. But basic research in the biomedical space is exploratory - many hypotheses are tested and preliminary conclusions are drawn... and many of those turn out to false leads/dead ends. Experts in basic and clinical research sort themselves out - map where knowledge stands - in review articles (secondary sources). Those reviews tend to talk about what hypotheses are looking to be true and productive; they don't waste time documenting all the dead ends. The literature is littered with basic research papers that turned out to be dead ends (not retracted -- not egregiously flawed or fraudulent science - but just dead ends) and those papers just sit there in the literature. You or I have no way of knowing which have results that the field has found true and productive, and which are not. So we avoid them like the plague. 3) we are an encyclopedia, not a newspaper, and not a cutting edge journal. Our mission is to give reliable information to the public. So we don't add content based on the latest hyped-in-the-media paper, and we try to hard to only include content that is supported by experts in the field, reviewing the field - that is how we ensure the content is reliable. Jytdog (talk) 20:16, 4 August 2014 (UTC)

Cancer Research UK Cancer Stats?

Do medical editors here have any particular views on the appropriateness or otherwise of using this internet service as a reliable medical source, especially when recent pertinent MEDRS are hard to come by (e.g. for coeliac disease as a putative risk factor for esophageal carcinoma - [1] vs. [2])? Although their Cancer Stats parent page gives the impression of being only UK, in reality this internally researched source of public information also provides some more global content. (ping Wiki CRUK John to alert him of this query) 86.157.144.73 (talk) 15:26, 11 August 2014 (UTC)

The service is reliable enough, but Wikipedia does not present statistics unless those statistics are themselves reported by a secondary source. There are some exceptions which are commonly reported in infoboxes. You are proposing to reference interpretations of the statistics. In my opinion, these kinds of states are often useful, but I have never seen these kinds of statements made often on Wikipedia so there may be other opinions if anyone tries doing this. I like the idea. Blue Rasberry (talk) 18:09, 2 September 2014 (UTC)

I don't see any inherent barrier to accepting these as valid MEDRS and using them ad lib. This is a large national organization and if we can reference one of their reports all is good. -- CFCF 🍌 (email) 05:46, 4 September 2014 (UTC)

The pages are nicely referred and if supported by a secondary source we can just use that. It gives us a better idea of the data of publication of the meta analysis or review in question. NCI is a decent source. Doc James (talk · contribs · email) (if I write on your page reply on mine) 06:41, 4 September 2014 (UTC)

Acupuncture source

Seeking comments re a 2011 review of reviews by Ernst (PMID 21440191). (This may have come up here before; I know it came up someplace, but I can't remember where.)

The debate is about how to interpret a particular sentence in context. It boils down to a right and wrong way to read a source and unfortunately requires more than just a glance (but not much more than, say, 10-15 minutes, depending on one's background). I've outlined the issue at User:Middle_8/Ernst-11. That should provide enough background, but I highly recommend also reading the paper if possible. Feel free to email me for a copy.

Note: Unfortunately, there's been some heat over there; hoping for lots of light! (And also hoping to keep this focused just on content and not on personalities.) Thank you very much for your time! --Middle 8 (POV-pushing • COI) 17:55, 2 September 2014 (UTC)

We've been through this before. I'd urge anyone interested to read Middle8's commentary at User:Middle_8/Ernst-11 and try to understand the point he's making: that Ernst has reached a conclusion (real and sham acu are equally effective), but because he refers to that conclusion in the context of some primary sources that he has reviewed, that means that we can't use the conclusion he states. This is completely against our policy of respecting secondary sources; none of us can know how many sources, primary and secondary, he has examined to reach his conclusion. The primaries he refers to are introduced specifically to refute the efficacy of acu in treating LBP and osteoarthritis (the only areas where his review of secondaries suggested showed efficacy beyond placebo), and we cannot baldly assume that these were the only sources that contributed to Ernst's stated conclusion. How many times do we need to state that Wiki editors are not qualified to interpret sources? - and that's why we are so insistent on using secondary sources for ourselves; we cannot make judgements on a respected author's choice of sources because his expertise is what we rely on. We should never be trying to substitute Middle8's interpretation of secondary source for what the source actually says.

We all know that 'real' acupuncture can have positive effects on pain; but we also know that there is no credible mechanism that could produce different results from 'sham' acupuncture. That the two are equally effective within the bounds of experimental error is obvious to anyone who does not believe in "qi" and "meridians" (in other words, the whole scientific community) and we should have no difficulty in stating that clearly within the article. --RexxS (talk) 18:46, 2 September 2014 (UTC)

A simpler summary: the question is, should we cite Ernst's statement "real acupuncture was no better than sham" as being the conclusion of his review per se, or not. The statement appears in the first paragraph on p. 762 of PMID 21440191.

In context, it is apparent that his statement refers to some recent, high-quality trials that showed null results. Contrary to RexxS's reading, it does not refer to the reviews he looked at, and is not a general conclusion of his review. The majority of those reviews were actually positive for acu, but with numerous contradictions and caveats. That's why he said those result should be seen in light of recent, better trials. The highlighted text at User:Middle_8/Ernst-11 illustrates this context, and the unlikelihood of RexxS's reading. --Middle 8 (POV-pushing • COI) 05:32, 3 September 2014 (UTC)

a more detailed version of Middle 8's arguments

The context of Ernst's comment (real and sham acu are equally effective) -- what specifically it was referring to -- is I hope apparent on reading the paper, and/or User:Middle_8/Ernst-11. He's referring to recent trials only, whose conclusions he contrasts with the 57 reviews he examined (which were mostly positive for acu, a result Ernst doubts will hold as studies get better). The literature on acu is more or less correctly summarized in the lede: no evidence of efficacy for most conditions, and inconsistent findings of smallish amounts of efficacy for pain and nausea (which may be due to artifact). (Nonetheless acupuncture is practiced at academic centers including Harvard, Yale and Stanford, which is prima facie evidence of a considerable degree of mainstream acceptance. I think the question, to them, is not efficacy vs. sham, but whether to refer: i.e. efficacy against standard care or waitlist.) Anyway, the current state of the field is what we seek to depict, and there is (obviously) mainstream debate both over its use in academic centers (which critics call "quackademic") and its efficacy against sham (which some reviews do find, and others dispute, and the discussion is vigorous). If there were unanimity that there is no efficacy -- which is what we would be stating, literally, by saying "real and sham acu are equally effective" -- then there would be no debate, no Novella and Gorski and Colquhoun making their arguments. Since the literature does include findings of efficacy (however disputed), how can "real and sham acu are equally effective" be an adequate summary of the literature? It's not -- nor is it how Ernst depicts the reviews he's examining. What it is, is how he depicts newer trials, which he believes herald a future consensus at which we have not (yet) arrived. -- Middle 8 (POV-pushing • COI) 21:45, 2 September 2014 (UTC)

It's clear to anyone who reads the article that "individualized acupuncture is not better in reducing symptoms than formula acupuncture or sham acupuncture with a toothpick that does not penetrate the skin" is Ernst et al.'s interpretation of Cherkin et al. (2009) and should only be quoted as such. The conclusion of Ernst et al. is the one in the abstract "In conclusion, numerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain. Serious adverse effects continue to be reported." Whether Ernst et al. were justified in reaching their conclusion isn't for us to evaluate in article space, whatever our personal opinions. Peter coxhead (talk) 08:40, 3 September 2014 (UTC)

While not the primary conclusion of the review of review (the study does not appear to be designed to answer the question of sham vs real and so cannot be said to be the primary conclusion), Ernst spends an entire paragraph (the largest paragraph in the discussion section, as a matter of fact) to basically endorse two high quality primary studies finding "real" wasn't different than "sham". My interpretation: It would be incorrect to say that discussion about sham acupuncture was the only or primary conclusion of the paper; however, does seem to be a conclusion Ernst is making and therefore citable as a source for that conclusion. Yobol (talk) 13:57, 4 September 2014 (UTC)

Meta comment: I saw a research report recently that said the involvement of experienced editors led to less collaborative editing in sexuality-related articles. I wonder if the same would be true for altmed. WhatamIdoing (talk) 01:13, 3 September 2014 (UTC)

Scary. I think alt-med wars are a partisan conflict that transcends experience level. Then again, experience might make it worse, insofar as it leads to more creative and subtle ways of being tendentious. In fact, there are some really good editors who can and do collaborate very well, but are discouraged by the tendentiousness. However, they can't always agree among themselves who are the worst offenders, so....

This reminds me of a comment I heard about why academics argue so intensely among themselves: because the stakes are so low (sic). "Someone is wrong on the internet." Arguing becomes a recreational activity, a way to blow off steam.--Middle 8 (POV-pushing • COI) 05:22, 3 September 2014 (UTC)

JzG/Guy has written or will write to Ernst himself to clarify, which should settle the issue, as long as it's handled precisely and transparently -- but this broad-brushed and dismissive comment does not exactly inspire confidence. I look forward to hearing the specifics. We can get both the letter and spirit of the law right without doing violence to either. That's the hallmark of a good encyclopedia. --Middle 8 (POV-pushing • COI) 05:04, 4 September 2014 (UTC) edited 07:06, 4 September 2014 (UTC)

I don't see how that will help at all. A response from the author by e-mail is irrelevant; it can't be used here. Our task is solely to represent, accurately, what the published source says, which is crystal clear in my view. Peter coxhead (talk) 08:56, 4 September 2014 (UTC)

(Middle 8 asked me to comment on this point; I am active at the Acupuncture page but had hitherto been avoiding the relevant talkpage discussion as having a poor heat to light ratio). Middle 8's reading of the source in context agrees with mine that the sentence "Real and sham acupuncture were both more effective in reducing pain than no acupuncture at all, but real acupuncture was no better than sham" is a report of the conclusion reached by Suarez-Almazor 2010. Ernst 2011 is pretty harsh on the quality of available evidence and should by no means be cited as supportive of the practice, but this particular statement is providing context for the results being reported: it is not a result itself of the present work. Being cited in this way does, however, provide a good argument for citing Suarez-Almazor 2010 itself for this conclusion (limited to knee osteoarthritis if we want to be very particular). - 2/0 (cont.) 15:12, 4 September 2014 (UTC)