Wikipedia talk:Identifying reliable sources (medicine)

Work priority?

Could we possibly prioritize three things, before we get bogged down?

1. Work on general better education and awareness of MEDRS.
2. Look at beefing up some of the wording now here along the lines of BLP to include more of "we need to get it right" kind of wording, regardless if it's policy.
3. Define our samples so we can begin to discuss what we think should be done in each case, akin to the possibilities we have with BLP.

I submit that if we can come close to agreement on individual samples (like the nine above), it will make it easier for us to figure out which way to go with the page. And I'd love to see us come up with a general educational blurb that we can post over to DYK, GAN, Articles for creation, AN, ANI, and any new article or editor that encourages more people to look at MEDRS. I've been patrolling new medicine articles lately, and what is out there is just scary-- particularly that so few at Articles for Creation are aware of MEDRS. We should strive to educate new editors early on ! And I'm finding new articles never tagged with the Medicine WikiProject, which is another way to educate about MEDRS. [1] Let's get 'em when they come in the door! User:AlexNewArtBot/MedicineSearchResult and Wikipedia:WikiProject Medicine/Article alerts. SandyGeorgia (Talk) 20:00, 29 February 2012 (UTC)

I think better education and awareness of MEDRS is a good idea, but... If achieving policy status at some time in the near future is a goal, grabbing a lot more eyeballs at this particular point might be counterproductive. Nathan ^T 03:42, 1 March 2012 (UTC)

On point #3, perhaps a "MEDRS examples" page would be a helpful way to provide practical advice. WhatamIdoing (talk) 23:35, 1 March 2012 (UTC)

Yes, but I still thought that wrt those examples, we are looking to develop something akin to BLP that gives guidance on what can of text can be "shot on sight", and how to handle the other nine examples. SandyGeorgia (Talk) 01:12, 2 March 2012 (UTC)

I created Template:RSPlease a while ago but not sure how to get the word out. What we need really is more editors to take care of WP:MEDRS compliance and to support each other in the issues that arise. --Doc James (talk · contribs · email) 11:51, 4 March 2012 (UTC)

Sorry I dropped out of the discussion for a bit - business trip to San Diego, about as far from home as I could possibly go and still be in the lower 48. Anyway, I think we should focus on #2 and #3. Crafting stricter language will be easier, at least until we get to describing the hierarchy of problematic content. There are a few approaches we could take here, some that might be really quite novel for Wikipedia. Maybe the most familiar to people from other settings would be a staged system with criteria - i.e. "Unsourced statement of a fact that is not considered to be common knowledge, with no potential for causing harm if relied upon as fact by a reader" for a minimal harm category, up to "Unsourced statement or assertion of fact capable of resulting in significant or severe harm if relied upon by a reader." I don't think we would want the bureaucratic nightmare of people tagging these unsourced statements according to tiered categories, but it could be a useful way of distinguishing the severity of identified problems to others (and, as a result, a handy tool in enforcement). Nathan ^T 17:41, 5 March 2012 (UTC)

I think this is an interesting idea; as a general rule though, I think the BLP mirror of "contentious" material could be applicable here. We don't want to be overzealous in removing unsourced/poorly sourced information, but "contentious" material, like non-mainstream "cures" or the like should be removed on sight unless strongly supported by a good source. Yobol (talk) 16:20, 8 March 2012 (UTC)

After I posted this I thought more about it, and I don't really think its workable. It relies too much on editors being able to determine the harm or "contentiousness" associated with particular assertions. A better method is to make a sort of dummy's guide to common errors, with appropriate responses - perhaps based on the examples Sandy listed above. I've started work on a first pass attempt at User:Nathan/MEDRS. Nathan ^T 16:46, 8 March 2012 (UTC)

Related discussion

Folks here could probably contribute usefully to the discussion at Wikipedia talk:Manual of Style/Chemistry#Proposal to rename Current Events and edit the section on Resources on primary sources. WhatamIdoing (talk) 17:58, 3 March 2012 (UTC)

Primary sources

As I seem to be getting into continual conflicts over the "true" intention of MEDRS with regard to primary sources, I think adding examples of when primary sources would be allowed would be helpful in these types of discussions (i.e. if no reviews are available, etc). Thoughts? Yobol (talk) 16:22, 8 March 2012 (UTC)

From my perspective, the intent of the guideline is to prevent the abuse (not the use) of primary sources. I'm fine with primary sources being used to illustrate current medical knowledge and understanding. However, there has been a longstanding issue with the abuse of primary sources, in which a few carefully selected primary sources are editorially highlighted to rebut expert opinion, or to give undue weight to a minoritarian viewpoint. In my mind, this guideline is intended to address the abusive scenario, but not to prohibit the use of primary sources where they help illustrate the state of modern medical knowledge.

I think we should be careful about specifying "rules" for when primary sources are OK, because my experience suggests that explicit rules are gold for wikilawyers. Instead, we should clearly articulate the principles and reasoning behind the cautions about primary sources, and leave the case-by-case interpretation to a consensus of clueful editors. Just my opinion. MastCell ^Talk 18:32, 8 March 2012 (UTC)

@MastCell: I agree, hard rules is not the way to go here, as I agree they would be wikilawyered, and frankly, there would probably be too many exceptions to any small set of rules to be useful. However, I think examples of when primary sources can/should be used will help for everyone involved (as we can see in this very thread). Yobol (talk) 01:40, 9 March 2012 (UTC)

I have a feeling I've misunderstood, but should we be using primary sources when no secondary sources are available? I would have thought that's the kind of situation that we ought to avoid. Primary sources can be useful for illustrating a point made by a secondary source, but without a secondary source we've no way to judge due weight, and articles/sections based on primary sources tend to be a mess of OR. Jakew (talk) 18:44, 8 March 2012 (UTC)

I think there are instances where primary sources are OK. For example, I started total body irradiation, which I think is an important, article-worthy topic. But secondary sources dealing specifically with TBI are limited or non-existent, so the article is mostly based on a summary of primary sources. I think this is OK because it's not a particularly controversial subject, I think there is little dispute about how to use or interpret those primary sources, and I think our article uses the primary sources to illustrate (rather than rebut or question) current medical opinion. But I think we do need to be very careful on more controversial articles or topics; I'm just wary of being too prescriptive or dogmatic since I think there are examples where we can write decent capsule articles using primary sources. MastCell ^Talk 18:57, 8 March 2012 (UTC)

I'm not convinced, MastCell. A quick Google Book search turned up Chapter 18 "Total Body Irradiation" in "The Physics of Radiation Therapy" by Kahn F. ISBN 0781788560. Also, Chapter 14 "Total Body and Hemibody Irradiation" of "Perez and Brady's Principles and Practice of Radiation Oncology" ISBN 078176369X. PubMed finds 39 reviews with "Total Body Irradiation" in the title. In addition, there are a number of patient-information booklets online that (despite being normally discouraged by MEDRS) could be used to flesh out the patient-experience aspect of this topic. So the best secondary sources are available for this topic, though of course individual editors may be restricted in their access to them.

Also, although the article cites some primary research papers, I suspect it isn't using all of them as primary sources. Rather, it is picking some material from the introductory discussion within the papers. It isn't clear that all of the text is sourced and the final paragraph on Ewing sarcoma uses dated sources and one wonders if it is even worth mentioning any more.

So, although a talented knowledgeable writer can piece together a short article worth reading after browsing some primary research papers, I wouldn't hold this up as an example of a good exception to MEDRS typical advice. Colin°^Talk 20:08, 8 March 2012 (UTC)

The reason I had added to policy that "material" based purely on primary sources should be avoided was to not only address the viewpoint of reliability but also subsequent notability as a matter of speaking. If a particular sub-topic within a larger article (or for that matter the article itself) hasn't been addressed by reliable, secondary sources, then there is a prevalent question on whether the sub-topic being reported has notability worthiness. Clearly, avoiding primary sources doesn't mean never use. But editors have to give a strong consideration to finding secondary sources when both reliability and notability of an article or material (however non-controversial) is dependent purely on primary sources. Wifione ^Message 03:26, 9 March 2012 (UTC)

One appropriate use of primary sources is to fill in the blanks. Any given subtopic might be so trivial from the perspective of high-quality secondary sources that they don't bother to mention it, but still might be important or expected by our readers. For example, our biographies include the birth year for the subject (when known) and the person's location/nationality per WP:OPENPARA, even though many secondary sources about people who are merely semi-famous won't mention these things. WhatamIdoing (talk) 05:52, 9 March 2012 (UTC)

I'll offer up an example for discussion. Klazomania was a stub when a class project took it on. There is one statement about it in one review, but it meets notability. The students expanded the stub based on primary sources. I cleaned it up, trying to leave the use of primary sources only for descriptions of symptoms. I've no attachment to the article, had to clean up after students who are likely no longer editing-- it might provide a sample for discussion. SandyGeorgia 03:41, 9 March 2012 (UTC)

That's a great article. Of the six sources, we have two reviews, one diagnostic manual, one medical dictionary and two examples of a "case report and review of the literature". Since there are only a few case reports, you've described those cases using the reports. We've also got a few hypotheses presented and they are attributed to their author's primary-source texts with both in-text attribution and footnote citations. That's the extent of primary-source usage and one that fits with policy. This is also a good example of when it is misleading to categorise a whole document as a "primary source". This is a topic that has few good texts to draw on and where editors have done a good job with what is available. Colin°^Talk 09:33, 9 March 2012 (UTC)

• Just as a matter of the actual policy, nobody should be saying that MEDRS prohibits using primary sources (as it sounds like Jakew was thinking). That's just untrue. If people are saying such things even after being reminded that it's untrue, that's a pretty serious problem. And before anyone jumps up to edit this page (or WP:PSTS) to change this, please let people know with a thread topic and probably a RfC. These pages are longstanding and have been in place for years, with a lot of debate. Discussions should be substantive rather than cursory link to WP:MEDRS; that would seem more like wikilawyering. We also need to remember the old adage that the perfect is the enemy of the good. Not all of us are right next to world-class medical research libraries. Medical review journals tend to be locked up pretty tight behind paywalls. That doesn't mean that people are prohibited from contributing. This is not Scholarpedia; it is the encyclopedia anyone can edit and everyone knows it and the limitations which that involves. II | (t - c) 05:52, 9 March 2012 (UTC)

Just as a matter of the actual policy, let's quote it:

Wikipedia articles should be based on reliable, published secondary sources and, to a lesser extent, on tertiary sources. Secondary or tertiary sources are needed to establish the topic's notability and to avoid novel interpretations of primary sources, though primary sources are permitted if used carefully. Material based purely on primary sources should be avoided. All interpretive claims, analyses, or synthetic claims about primary sources must be referenced to a secondary source, rather than to the original analysis of the primary-source material by Wikipedia editors.

Ok, so there's a difference between "prohibited" and "generally unwise" but let me tell you that every time I've seen someone argue that "... says you can use primary sources" they have been trying to do something wrong. To me, it is like driving on the correct side of the road. There are times when you need to drive on the other side, but you don't base your driving style on it. If you find yourself getting into an argument over using primary sources to present some fact or cover some topic, then it is likely you are on the wrong side. Like WhatAmIDoing's favourite "challeged" aspect to sourcing, nobody is challenging Sandy's text at Klazomania and I'm quite sure she's more than capable of gleaning something good from inferior material. There is a spectrum of source qualities and few sources are absolutely prohibited. We don't encourage the use of primary research papers, per policy, just like we don't encourage the use of abstracts or newspapers or press releases. There are times when these source types are helpful, mostly where the topic isn't well documented. I agree with WhatAmIDoing's "fill in the blanks" point. For example, in the total body irradiation article, look at this patient information leaflet. It is full of useful information that I bet nobody would ever mention in a review or a chapter in a medical text. I wouldn't use it to list the indications or effectivenes for various conditions, but we can use it to fill in the blanks. For the big article topics like obesity, cancer, epilepsy one should be easily capable of writing all that needs to be said using secondary sources. Colin°^Talk 09:03, 9 March 2012 (UTC)

I believe that the patient information leaflet would be considered a secondary source for information about TBI. Secondary is not another way of spelling "high-quality".

I am happy to use such "low-quality" (properly, non-academic) sources to support very general or basic information. I think it's good for readers to find a variety of types of sources, so that people who don't have a scholarly bent can find sources that are appropriate to them. WhatamIdoing (talk) 16:05, 9 March 2012 (UTC)

Sorry if that was confusing. I wasn't saying the PI leaflet was a primary source, merely that it was one of the types of sources we wouldn't recommend folk base an article on, like abstracts, and press releases, etc. Sometimes, you have to make do with what you can get. And sometimes, an atypical source can be exactly what you need: it is very difficult to get academic sources that discuss "what the patient goes through". Colin°^Talk 19:25, 9 March 2012 (UTC)

WebMD as a reliable source

The current guidelines explicitly name WebMD, stating that WebMD and other sources "are usually acceptable sources in themselves". I'd like to bring this into question, and propose removing WebMD from the list or changing the wording here. Lately I've been delving into more depth in material on which I am fairly knowledgeable, and finding serious problems with WebMD's research. Let me give two examples:

• The following page on WebMD, [2], says "One study has found regular use of products containing tea tree oil may cause gynecomastia", and then cites the following study: [3], which, if you read the source, provides insufficient evidence to support the claim made by WebMD. The way WebMD has cited the source suggests they merely read the headline of the article or possibly the abstract, rather than actually reading the article.

• The following page on WebMD: [4] makes the claim "black tea has the highest caffeine content", without citing any source. This is a topic I have researched extensively, as I maintain a page on RateTea about the caffeine content of tea. In particular, the following sources: [5] and [6] draw this into question. Also, although it's less relevant because it's not really a medical topic, the page also contains information about tea that is sloppy, i.e. that Pu-erh is "Considered a black tea", and that Wuyi is "One variety of oolong" (it is a region that produces oolong and other teas, not usually considered a variety).

Furthermore, I have contacted WebMD about the caffeine point, and I did not receive any response, nor has it been corrected. These may be only two examples, but they're examples of topics that I have researched the most thoroughly and know the most about. I suspect that if I really delved into depth, I would be able to find more examples of how the quality of scholarship on this site is lacking and is thus unsuitable for citing as a reliable source.

My questions are: do you think WebMD's quality is lacking relative to the other sources given? If so, we could remove it from the list. Or do you think that the quality of all these sources are relatively similar? In this case, it might be better to change the wording to "may be acceptable..." rather than "are usually acceptable..." Cazort (talk) 16:23, 3 April 2012 (UTC)

I have seen at least two instances of poor analysis and conclusions published in Cochrane Review and it is considered one of the highest standard sources. WebMD I imagine given it's size and scope has lots of health-care professionals writing for it, so some mistakes are inevitable. From my use of WebMD, they seem fairly comprehensive and accurate. If you see a clear error in WebMD, find a different good source to add a differing viewpoint or possibly replace the WebMD with it if it is a clear error on WebMD's part. I think removing WebMD as a recommended source is like basically throwing the baby out with the bathwater and will create more problems than it solves. Others may have a differing viewpoint though.--Literaturegeek | T@1k? 16:48, 3 April 2012 (UTC)

Hmm, you're right that even the best sources have problems. My impression of WebMD is an overall impression, not just based on these specific examples. My impression of WebMD is that it tends to present material that fits with societal "common knowledge", sometimes perpetuating myths without questioning, and that it does not show enough evidence of critical research for me to be comfortable with it being mentioned in the guidelines as it is here. I personally have a more favorable impression of Mayo Clinic, but I don't feel confident with my impression, it just is an intuition, and I have no opinion on the other sites mentioned. "Throwing the baby out with the bathwater" is less of a concern for me in this case because I don't find much unique material on WebMD, i.e. material that can't be found in higher-quality sources. If we leave WebMD and the other sources, I'd be more comfortable if we weakened the language to "may be acceptable..." or something similar. Cazort (talk) 18:19, 3 April 2012 (UTC)

I pretty much agree with LG: all of these are "usually acceptable", by which we mean that they are "sometimes not". Like any source, if they disagree with the general run of sources, or with sources that are higher quality and/or more appropriate (e.g., a source entirely about tea-growing regions for your Wuyi example), then they should be avoided. WhatamIdoing (talk) 19:22, 3 April 2012 (UTC)

I rarely use WebMD. There are concerns of conflict of interest per [7] but it is not the only source with one. While there are better sources I would not argue for an outright ban. Doc James (talk · contribs · email) 01:53, 4 April 2012 (UTC)

Hmm. I also would not propose an outright ban yet, but after reading this article, I'm feeling stronger about changing how we mention WebMD, relative to, say, Mayo Clinic (I haven't found any overt misinformation on Mayo Clinic, whereas I've pretty consistently found misinformation on WebMD when I've delved deep enough to assess the truth of the material there). I've had an impression that Mayo was more reliable and it's seeming an increasingly poor choice to leave them in a list that implies they're roughly on the same level in terms of reliability. I have not had much experience with the other sites in the list so I don't really have any opinion on them at the moment. Regarding wording, I think there's a huge difference in connotation between saying something is "usually acceptable" or "sometimes acceptable". My personal inclination would be to consider WebMD to be "sometimes acceptable". For instance, I tend to be very skeptical of its recommendations about drugs too. For example, it has a ton of pages on Benzodiazepines which implies they are suitable for long-term or general use, like this page that provides it as an option for bipolar disorder: [8], yet there seems to be a growing medical consensus that these drugs are only suitable for short term use for acute anxiety. Mayo Clinic expresses this: [9] "Benzodiazepines are generally only used for relieving acute anxiety on a short-term basis". The potential conflict of interest, which I did not know about until reading that article, would make me more concerned particularly because of this sort of discrepancy. Cazort (talk) 14:15, 6 April 2012 (UTC)

"Unless the primary source itself directly makes such a claim"

The sentence "Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources" is wonderfully clear, and I've been wondering whether to incorporate it in WP:PSTS (on grounds that it's a logical consequence of existing policy, and hence applies to all articles). However, the final part of the sentence ("unless the primary source itself directly makes such a claim") doesn't make sense to me. If a source debunks, contradicts, or otherwise comments on the conclusions of a secondary source then it isn't a primary source. It's a secondary (or tertiary) source. What is it intended to mean, and how can it be rephrased to better express that meaning? Jakew (talk) 09:58, 24 April 2012 (UTC)

Agree we should get rid of it. --Doc James (talk · contribs · email) 10:37, 24 April 2012 (UTC)

(outdent) It means, if a primary source directly criticises a secondary source, for example, a primary source, may point out undeclared conflicts of interests or perceived flawed methodology used by reviewers or confounding variables etc. The primary source would basically be saying the review by such and such group of researchers is flawed because,,,,. The primary source would have to specifically mention a secondary source by name. It is NOT the same as using a primary source such as an individual study to debunk a review of studies. I don't think the line should be got rid of but perhaps it is possible to better word it?--Literaturegeek | T@1k? 23:55, 24 April 2012 (UTC)

But many of these comments are not WP:DUE. For example we have this report by the AHRQ [10] which was criticised by one of the leaders of the religious movement in question [11] --Doc James (talk · contribs · email) 06:10, 25 April 2012 (UTC)

Ah. As I see it, whether a source is primary or secondary depends on how it is used. If we cite a source for its criticism of another source, then we aren't using it as a primary source. We're using it as a secondary source. That's why I find it really confusing to see it described as a "primary" source. I think we've got used to thinking "primary source = anything in a primary research study, secondary source = anything in a review", which is often convenient, but it's an oversimplification. As Doc James points out, though, there are frequently undue weight issues associated with citing primary research studies.

How about rephrasing: Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources. Explicit criticism of secondary sources, however, may be included if appropriate, but be sure to assess due weight. Jakew (talk) 07:32, 25 April 2012 (UTC)

I don't agree with the suggested "Explicit criticism of secondary sources, however, may be included if appropriate". We really should try to mostly not discuss the sources in article text at all. Sometimes it is useful to explicitly mention research studies and meta analyses and even literature reviews, in text, but generally we should be in the business of describing facts, not the means by which folk discovered or write about those facts. So I don't want to encourage editors to argue/prove a point in front of the readers.

Note that the guideline text does not say the primary source debunks the text of the secondary source, or the methodology or even the moral character of the authors (that would, indeed, make it a secondary source on the review it was debunking. It says it explicitly debunks or contradicts the conclusions in the secondary source. I think what we're trying to avoid is something like this garbage: "A Cochrane review in 2006 concluded that drinking XXX fruit juice had no protective benefits on YYY cancer. However, recent research shows that XXX fruit juice contains aaa compound [link to research study on fruit juice composition], which is protective against cancer [link to research study injecting aaa compound into rats]. Neither primary research paper directly or explicitly contradicts the conclusions of the review. However, if somone did a long-term study of people who regularly drink XXX fruit juice and those who didn't, and found and stated that it did indeed appear to have protective benefits on YYY cancer.... That recent research could potentially be used to debunk the old review. It would be explicitly debunking/contradicting the conclusions, but doesn't even need to mention previous reviews, so doesn't become a secondary source. Colin°^Talk 12:04, 25 April 2012 (UTC)

When we talk about directly making the claim that it de-bunks older works, I think we are looking for a paper that says something like, "The received wisdom in the field (as seen in every textbook for the last 100 years) says that patients should not be permitted to eat anything for 24 hours after major surgery. However, as far as we can make out, this popular old idea is based on zero evidence, and we've actually done a proper randomized, controlled trial, which we outline here, and the data produced says that the old surgeons' ideas about low diet is a bunch of bunk."

Our approach to using them (the textbooks and the RCT paper) would then look something like "Keeping the patient on a low diet after major surgery has been widely recommended,^[1][2] but newer research suggests that it may be unnecessary.^[3]"

If, on the other hand, we had the same study, but the authors made no direct mention of how their study supersedes the older works, then we would probably ignore it (for now): "Keeping the patient on a low diet after major surgery has been widely recommended.^[1][2]" WhatamIdoing (talk) 13:51, 25 April 2012 (UTC)

I disagree. I don't think there is any requirement for the new source text to explicitly name the debunked sources or to imply them with a remark like "every textbook for the last 100 years". The "de-bunks older works" comment falls into the same trap of confusing source text/work with the facts it presents. Criticism of older "works" might happen in a field like history, say. But research in medicine would tend to overturn previous "facts" rather than "works". Also, there's nothing the research authors can say or not say that makes any difference to the WP:WEIGHT we give to their findings. So the argument that, depending on what they say, we could ignore them or not, doesn't work. The guideline we are discussing is preventing original research based on primary sources. It is a separate issue to consider whether that primary research paper's conclusions have enough weight to warrant mention. Colin°^Talk 15:22, 25 April 2012 (UTC)

I have to say, I'm not comfortable with citing primary sources in either situation.

In the case that the authors assert that their study is more significant than the sum of previous work, their assessment is unlikely to be wholly objective. They may well be correct, but it wouldn't be surprising if their view of the importance of their work was a little out of proportion. We're all prone to thinking our work is more important than it really is. I would generally think it best to wait until a secondary source cites their paper and judges the significance of their work from a less biased perspective.

In the case that they make no particular assertion, I'd be very concerned if editors were to make that assessment themselves. True, much of the time the decision will be rational and in accordance with principles of evidence-based medicine, but many controversial topics are subject to widespread POV pushing, and it would be best to avoid long-winded debates full of original research about the relative importance of various studies. Again, why not just wait until a secondary source becomes available? It's unlikely to be that urgent, after all... Jakew (talk) 09:35, 27 April 2012 (UTC)

MEDRS

I think the guidelines has become too complicated for its own good, specifically about primary sources. For example this vital article http://news.harvard.edu/gazette/2006/12.07/11-dairy.html would be valid in most places but not here, this really concerns me and shows the flaws of Wikipedia and the urgent need for revision of the guidelines in general of any field. Helios solaris 16:11, 26 April 2012 (UTC)

It is complex because the problem is complex. The article you link is a newspaper/magazine type of article based on a talk someone gave in their lunch hour. Why do you think a serious encyclopaedia should base its articles on that kind of "I've got a radical new idea and have cherry-picked a bunch of studies that support it" kind of journalism. -- Colin°^Talk 16:36, 26 April 2012 (UTC)

Because if not it will fall behind. Helios solaris 17:33, 26 April 2012 (UTC)

I don't mind "falling behind" speculation and cherry-picking. The researcher in your source says that her hypothesis is unproven and not ready for prime time.

That said, this source is usable, for certain limited purposes. You could use it, for example, to support a claim that research is being conducted on the relationship between dairy consumption and hormone-sensitive cancers. You can't use it for a claim that any such link has been proven, or that readers should change their diets. WhatamIdoing (talk) 19:04, 26 April 2012 (UTC)

There is insufficient evidence of WP:DUE. We have much better sources discussing what research is ongoing. --Doc James (talk · contribs · email) 10:19, 27 April 2012 (UTC)

When you realise how hard it is to find secondary sources in the way that pleases Wikipedia and add it, even then it's suppressed, it becomes obvious that they are suspiciously superfluous. They exist only to subdue true evidence and strenghten the corporate establishment, just remember the tobacco industry do I need to say more. Critical thinking is clearly not welcomed here. --Helios solaris 16:25, 27 April 2012 (UTC)

There's nothing wrong with critical thinking, we just don't let it affect our edits. WP:NOR is long established policy for one very good reason: when "anyone can edit", some anonymous/pseudonymous editors will be incapable of critical thinking. If we allow it for you, we have to allow it for everyone, including those who can't think at all. Instead, we insist on wp:verifiability and secondary reliable sources. It works, though we have to keep explaining why we do it. LeadSongDog come howl! 19:34, 27 April 2012 (UTC)

Section tagging

Various templates (such as {{MEDCN}}, {{MEDRS}}) are available to tag reliable source problems inline. Is there one to flag up a whole section, i.e. some "medical source" version of {{Refimprove}}? If not, I think there should be. Peter coxhead (talk) 10:34, 1 May 2012 (UTC)

Sounds like a good idea. --Doc James (talk · contribs · email) 10:36, 1 May 2012 (UTC)

Well, if there isn't one already, how about something like my draft at User:Peter coxhead/Test/T1? As with {{refimprove}}, on which it is based, using the 1st parameter |section will replace "article" by "section". Peter coxhead (talk) 11:12, 1 May 2012 (UTC)

{{refimproveMED}}, though I don't know whether it has a section parameter. Nikkimaria (talk) 13:30, 1 May 2012 (UTC)

{{Medref}} (where did that refimprove come from and which do we prefer), but no section parameter. SandyGeorgia (Talk) 13:32, 1 May 2012 (UTC)

Neither of these have section parameters, although they could be added. {{RefimproveMED}} doesn't do what I want, because it says that the article is a health or medicine one, whereas I want a template to be added to e.g. a plant article that has a section on its supposed health benefits. {{Medref}}, too, suggests more to me that the whole article is a medical one; I'm not sure that WikiProject Medicine would really want to get involved in sorting out a few odd herbal medicine claims added to what is mainly a plant article.

But I don't want to create yet another template unnecessarily, so what do others think? Could {{Medref}} be worded so as to be a bit more appropriate to a non-medical article? I do think it's worth including this quotation "Biomedical information in articles must be based on reliable, third-party, published sources and accurately reflect current medical knowledge" especially in a warning meant for a non-medical article. Editors who add this kind of information to plant articles simply don't know about this requirement. Peter coxhead (talk) 14:23, 1 May 2012 (UTC)

Medref now has a section parameter: {{medref|date=May 2012|small=1}}. Cheers, — Bility (talk) 16:27, 1 May 2012 (UTC)

• Medref should now be appropriate for use on non-medical articles, too. The wording settled on ("This article needs more medical references for verification.") and template doc I wrote ("This template is intended to be placed at the top of articles with medical or health content") should cover health sciences related content in an otherwise non medical article. Further tweaks can be made if necessary. It now has a reason parameter |reason=foo is bar for giving specific details. Thanks, --92.6.211.228 (talk) 18:15, 1 May 2012 (UTC)

Actually {{Medref}} doesn't have a section parameter. This line:

| text = This article '''needs more [[Wikipedia:Identifying reliable sources

needs article replaced by {{#if:{{{1|}}}|{{{1}}}|article}}

exactly as in {{refimprove}}, {{no footnotes}}, {{more footnotes}}, {{how}}, etc. In all of these, if you use {{TEMPLATE-NAME|section}} the message becomes "This section needs more ..." instead of "This article needs more ..." Peter coxhead (talk) 21:57, 1 May 2012 (UTC)

It does. Sort of. If you look at the testcases (down the bottom), you can see it's different to those templates in that the small/section version doesn't say "This article" or "This section". --92.6.211.228 (talk) 22:10, 1 May 2012 (UTC)

(edit conflict) It's not a parameter called section that I meant, but a parameter you can use for the small version in sections, which is what I thought you guys were talking about. Right now the template uses this first unnamed parameter as the rationale, so you don't have to explicitly use |reason=. We can change this, although if you want to use the small version in a section, it's a moot point since that line won't be displayed. This would also mean any existing templates using the first unnamed parameter for their rationales would need to be updated to use |reason=. — Bility (talk) 22:36, 1 May 2012 (UTC)

Added it. Hmm. It replaces the word article with the reason value when all params are used. That's fixable of course. Did you want it to replace |small? Allow both? --92.6.211.228 (talk) 22:27, 1 May 2012 (UTC)

Add to Multiple issues?

I think it might be worth adding it to {{Multiple issues}} in line with other standalone templates. Wording (examples) needs to be agreed on.

Something like:

It needs additional medical references for verification. or

It relies on references which may not be reliable sources for medical or health content. or

It includes attribution to sources which may not be reliable for medical content.

Thoughts? --92.6.211.228 (talk) 18:45, 1 May 2012 (UTC)

Yeah, I was planning on getting it set up along with expert-subject, which is also missing, since I've already added medref to the multiple issues template on a couple articles. EDIT: Actually expert-subject is in there, just called "expert". Anyway, I'll go add an edit request for medref. — Bility (talk) 20:52, 1 May 2012 (UTC)

The guideline should be changed

"Basic advice", subsection "Assess evidence quality", paragraph 4 includes this statement: " "Assessing evidence quality" means that editors should determine the quality of the type of study. Editors should not perform a detailed academic peer review. Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions. "

Recently, this study was proposed as a reference for the article "Chemotherapy". Several editors (including, latterly, myself) were critical of the study and objected to the use of the study as a reference. See full details here.

WhatamIdoing quoted the above text from WP:MEDASSESS as justification for inclusion of Morgan's paper as a reference. In my opinion, the text fails to take into account those rare secondary sources that are severely biased or flawed, such as Morgan's paper.

I see no reason why editors shouldn't perform detailed peer reviews if they so choose. Moreover, I believe that such reviews can be helpful in determining suitability as references for Wikipedia. Axl ¤ [Talk] 19:41, 1 May 2012 (UTC)

Are there other reviews that are more recent that contradict this paper from 2004? We do say evidence from the last 3-5 years. --Doc James (talk · contribs · email) 20:28, 1 May 2012 (UTC)

I don't believe that there are any other reviews of this nature – because the whole premise of the review is flawed. Axl ¤ [Talk] 20:31, 1 May 2012 (UTC)

For information: the disputed text was added on 27 March 2010 by WhatAmIDoing. A talk page note at the time of the edit did not bring about any further discussion. However, the note refers to this earlier discussion and the whole section was added by Eubulides along with this discussion. -- Colin°^Talk 20:36, 1 May 2012 (UTC)

I think we should allow editors to exercise judgement in deciding whether a secondary source is actually suitable. There are loads of secondary sources that are of insufficient quality. In that sense, the "peer review" should assess the quality of the source. Disputes as to the suitability should be resolved by consensus. In this case, there was consensus that the source was not suitable. JFW | T@lk 21:46, 1 May 2012 (UTC)

Yes agree that consensus should be followed over the guideline. If the majority of editors do not see this ref as suitable than it should not be used. Does not necessitate a change in WP:MEDRS though. Many people still fell that WP:MEDRS says using primary sources is fine when high quality secondary sources are available which IMO is a more pressing issue.Doc James (talk · contribs · email) 21:54, 1 May 2012 (UTC)

I share some of WhatAmIDoing's concerns (expressed elsewhere) that we have to come up with rules that work for the POV pusher and the undereducated. I'm one of the undereducated. My brain is much smaller than that of MastCell and Axl and WhatAmIDoing; I have no health training beyond a first aid badge when I was in the Scouts; I'm just someone's dad. Some of WP's rules are here purely because of the special situation with our authors: we don't really know who they are; we don't trust who they say they are; most of them are bonkers; some of them aren't geniuses. An editor who is an expert in the subject is likely to find some of these rules frustrating at times.

The problem with the Chemotherapy paper raises two issues. The first is that academic papers, whether primary research or reviews, are written for a certain learned audience, and not for general consumption. That PubMed has made them (and particularly their abstracts) so accessible, is wonderful but at the same time very dangerous. As the article talk page shows, the abstract is over-simplified to the point of being misleading. And it takes someone quite knowledgeable to discover what the analysis actually looked at and is capable of saying. Combine this with the possibility that the paper is flawed/biased and we have a problem. Fortunately, it is not a common problem, and is less of a problem than editors trying to pick/review primary research papers.

The second issue is the level of source we are using here. There's too much emphasis on review papers among WP:MED folk in my opinion. Yes, it is wonderful to be able to read all this material on your browser or PDF on the computer, and search it and collate it on your hard disk. I do so myself. But what about books, guys, books! Chemotherapy is a big subject. I just searched Amazon on it and turned up several suitable texts including ISBN 160831782X: "Handbook of Cancer Chemotherapy", edited by Roland T. Skeel, Samir Khleif, published 2011 (an earlier edition is actually used by the article for one point, but sadly the page number isn't given). This is 832 pages of wonderful readable and up-to-date-enough material. Approximately 300,000 words by a rough estimate. Now our Chemotherapy article is about 4,000 words. Which means you could condense that book 75x to achieve the current text. That's about 5 words from every page.

The chemotherapy article is crap. There are many medical articles on WP I'd recommend to a friend, but that one isn't. Not by a long way. Why are we wasting days discussing the merits or otherwise of some 8-year-old review paper. Visit your university library bookshelves, or order a book (second-hand) from Amazon/etc -- you can always sell it later.

WP:WEIGHT is underused in sourcing disputes. Rather than wikilawering over PSTS or disecting the journal paper or whatever, just find out "What do experts generally say, in published reliable sources, when writing about this subject". We are used to using WEIGHT to judge research findings: we read and use reviews/etc to find that. Similarly, we should use WEIGHT to judge analysis reports/reviews, especially if editors dispute them. What do our serious weighty oncology textbooks have to say on the subject? Rather than fall out with each other over such things, find some expert person/body who has already thought hard about it, published their work, and use that. WP:WEIGHT specifically warns against editors forming their own consensus about what issues/facts to include in an article, so I disagree with JFW/Doc James on that point -- however I suspect if you follow my advice, the review and its conclusions would be rejected anyway. Colin°^Talk 22:09, 1 May 2012 (UTC)

I feel that, in this instance, the issue could be resolved by proper usage of existing guidelines. Colin suggested above several alternatives to giving infinite weight to a single source that is seemingly flawed. I briefly proposed another on Talk:Chemotherapy that involves exercising judgment not on the quality of the source, but whether the wiki text correctly reflect the conclusions being drawn by Morgan et al. To wit, is the source saying what the article is saying? In my mind, the source was misused because it was made to support a statement that is too broad. Unless a more systemic problem with WP:MEDASSESS is identified, I do not support making drastic changes to guidelines serving a useful purpose. Wafflephile (talk) 23:30, 1 May 2012 (UTC)