Wikipedia talk:Identifying reliable sources (medicine)/Archive 6

This is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 1

←

Archive 4

→

Work priority?

Could we possibly prioritize three things, before we get bogged down?

Work on general better education and awareness of MEDRS.
Look at beefing up some of the wording now here along the lines of BLP to include more of "we need to get it right" kind of wording, regardless if it's policy.
Define our samples so we can begin to discuss what we think should be done in each case, akin to the possibilities we have with BLP.

I submit that if we can come close to agreement on individual samples (like the nine above), it will make it easier for us to figure out which way to go with the page. And I'd love to see us come up with a general educational blurb that we can post over to DYK, GAN, Articles for creation, AN, ANI, and any new article or editor that encourages more people to look at MEDRS. I've been patrolling new medicine articles lately, and what is out there is just scary-- particularly that so few at Articles for Creation are aware of MEDRS. We should strive to educate new editors early on ! And I'm finding new articles never tagged with the Medicine WikiProject, which is another way to educate about MEDRS. [1] Let's get 'em when they come in the door! User:AlexNewArtBot/MedicineSearchResult and Wikipedia:WikiProject Medicine/Article alerts. SandyGeorgia (Talk) 20:00, 29 February 2012 (UTC)

I think better education and awareness of MEDRS is a good idea, but... If achieving policy status at some time in the near future is a goal, grabbing a lot more eyeballs at this particular point might be counterproductive. Nathan^T 03:42, 1 March 2012 (UTC)

On point #3, perhaps a "MEDRS examples" page would be a helpful way to provide practical advice. WhatamIdoing (talk) 23:35, 1 March 2012 (UTC)

Yes, but I still thought that wrt those examples, we are looking to develop something akin to BLP that gives guidance on what can of text can be "shot on sight", and how to handle the other nine examples. SandyGeorgia (Talk) 01:12, 2 March 2012 (UTC)

I created Template:RSPlease a while ago but not sure how to get the word out. What we need really is more editors to take care of WP:MEDRS compliance and to support each other in the issues that arise. --Doc James (talk · contribs · email) 11:51, 4 March 2012 (UTC)

Sorry I dropped out of the discussion for a bit - business trip to San Diego, about as far from home as I could possibly go and still be in the lower 48. Anyway, I think we should focus on #2 and #3. Crafting stricter language will be easier, at least until we get to describing the hierarchy of problematic content. There are a few approaches we could take here, some that might be really quite novel for Wikipedia. Maybe the most familiar to people from other settings would be a staged system with criteria - i.e. "Unsourced statement of a fact that is not considered to be common knowledge, with no potential for causing harm if relied upon as fact by a reader" for a minimal harm category, up to "Unsourced statement or assertion of fact capable of resulting in significant or severe harm if relied upon by a reader." I don't think we would want the bureaucratic nightmare of people tagging these unsourced statements according to tiered categories, but it could be a useful way of distinguishing the severity of identified problems to others (and, as a result, a handy tool in enforcement). Nathan^T 17:41, 5 March 2012 (UTC)

I think this is an interesting idea; as a general rule though, I think the BLP mirror of "contentious" material could be applicable here. We don't want to be overzealous in removing unsourced/poorly sourced information, but "contentious" material, like non-mainstream "cures" or the like should be removed on sight unless strongly supported by a good source. Yobol (talk) 16:20, 8 March 2012 (UTC)

After I posted this I thought more about it, and I don't really think its workable. It relies too much on editors being able to determine the harm or "contentiousness" associated with particular assertions. A better method is to make a sort of dummy's guide to common errors, with appropriate responses - perhaps based on the examples Sandy listed above. I've started work on a first pass attempt at User:Nathan/MEDRS. Nathan^T 16:46, 8 March 2012 (UTC)

Related discussion

Folks here could probably contribute usefully to the discussion at Wikipedia talk:Manual of Style/Chemistry#Proposal to rename Current Events and edit the section on Resources on primary sources. WhatamIdoing (talk) 17:58, 3 March 2012 (UTC)

Primary sources

As I seem to be getting into continual conflicts over the "true" intention of MEDRS with regard to primary sources, I think adding examples of when primary sources would be allowed would be helpful in these types of discussions (i.e. if no reviews are available, etc). Thoughts? Yobol (talk) 16:22, 8 March 2012 (UTC)

From my perspective, the intent of the guideline is to prevent the abuse (not the use) of primary sources. I'm fine with primary sources being used to illustrate current medical knowledge and understanding. However, there has been a longstanding issue with the abuse of primary sources, in which a few carefully selected primary sources are editorially highlighted to rebut expert opinion, or to give undue weight to a minoritarian viewpoint. In my mind, this guideline is intended to address the abusive scenario, but not to prohibit the use of primary sources where they help illustrate the state of modern medical knowledge.

I think we should be careful about specifying "rules" for when primary sources are OK, because my experience suggests that explicit rules are gold for wikilawyers. Instead, we should clearly articulate the principles and reasoning behind the cautions about primary sources, and leave the case-by-case interpretation to a consensus of clueful editors. Just my opinion. MastCell ^Talk 18:32, 8 March 2012 (UTC)

@MastCell: I agree, hard rules is not the way to go here, as I agree they would be wikilawyered, and frankly, there would probably be too many exceptions to any small set of rules to be useful. However, I think examples of when primary sources can/should be used will help for everyone involved (as we can see in this very thread). Yobol (talk) 01:40, 9 March 2012 (UTC)

I have a feeling I've misunderstood, but should we be using primary sources when no secondary sources are available? I would have thought that's the kind of situation that we ought to avoid. Primary sources can be useful for illustrating a point made by a secondary source, but without a secondary source we've no way to judge due weight, and articles/sections based on primary sources tend to be a mess of OR. Jakew (talk) 18:44, 8 March 2012 (UTC)

I think there are instances where primary sources are OK. For example, I started total body irradiation, which I think is an important, article-worthy topic. But secondary sources dealing specifically with TBI are limited or non-existent, so the article is mostly based on a summary of primary sources. I think this is OK because it's not a particularly controversial subject, I think there is little dispute about how to use or interpret those primary sources, and I think our article uses the primary sources to illustrate (rather than rebut or question) current medical opinion. But I think we do need to be very careful on more controversial articles or topics; I'm just wary of being too prescriptive or dogmatic since I think there are examples where we can write decent capsule articles using primary sources. MastCell ^Talk 18:57, 8 March 2012 (UTC)

I'm not convinced, MastCell. A quick Google Book search turned up Chapter 18 "Total Body Irradiation" in "The Physics of Radiation Therapy" by Kahn F. ISBN 0781788560. Also, Chapter 14 "Total Body and Hemibody Irradiation" of "Perez and Brady's Principles and Practice of Radiation Oncology" ISBN 078176369X. PubMed finds 39 reviews with "Total Body Irradiation" in the title. In addition, there are a number of patient-information booklets online that (despite being normally discouraged by MEDRS) could be used to flesh out the patient-experience aspect of this topic. So the best secondary sources are available for this topic, though of course individual editors may be restricted in their access to them.

Also, although the article cites some primary research papers, I suspect it isn't using all of them as primary sources. Rather, it is picking some material from the introductory discussion within the papers. It isn't clear that all of the text is sourced and the final paragraph on Ewing sarcoma uses dated sources and one wonders if it is even worth mentioning any more.

So, although a talented knowledgeable writer can piece together a short article worth reading after browsing some primary research papers, I wouldn't hold this up as an example of a good exception to MEDRS typical advice. Colin°^Talk 20:08, 8 March 2012 (UTC)

The reason I had added to policy that "material" based purely on primary sources should be avoided was to not only address the viewpoint of reliability but also subsequent notability as a matter of speaking. If a particular sub-topic within a larger article (or for that matter the article itself) hasn't been addressed by reliable, secondary sources, then there is a prevalent question on whether the sub-topic being reported has notability worthiness. Clearly, avoiding primary sources doesn't mean never use. But editors have to give a strong consideration to finding secondary sources when both reliability and notability of an article or material (however non-controversial) is dependent purely on primary sources. Wifione ^Message 03:26, 9 March 2012 (UTC)

One appropriate use of primary sources is to fill in the blanks. Any given subtopic might be so trivial from the perspective of high-quality secondary sources that they don't bother to mention it, but still might be important or expected by our readers. For example, our biographies include the birth year for the subject (when known) and the person's location/nationality per WP:OPENPARA, even though many secondary sources about people who are merely semi-famous won't mention these things. WhatamIdoing (talk) 05:52, 9 March 2012 (UTC)

I'll offer up an example for discussion. Klazomania was a stub when a class project took it on. There is one statement about it in one review, but it meets notability. The students expanded the stub based on primary sources. I cleaned it up, trying to leave the use of primary sources only for descriptions of symptoms. I've no attachment to the article, had to clean up after students who are likely no longer editing-- it might provide a sample for discussion. SandyGeorgia 03:41, 9 March 2012 (UTC)

That's a great article. Of the six sources, we have two reviews, one diagnostic manual, one medical dictionary and two examples of a "case report and review of the literature". Since there are only a few case reports, you've described those cases using the reports. We've also got a few hypotheses presented and they are attributed to their author's primary-source texts with both in-text attribution and footnote citations. That's the extent of primary-source usage and one that fits with policy. This is also a good example of when it is misleading to categorise a whole document as a "primary source". This is a topic that has few good texts to draw on and where editors have done a good job with what is available. Colin°^Talk 09:33, 9 March 2012 (UTC)

Just as a matter of the actual policy, nobody should be saying that MEDRS prohibits using primary sources (as it sounds like Jakew was thinking). That's just untrue. If people are saying such things even after being reminded that it's untrue, that's a pretty serious problem. And before anyone jumps up to edit this page (or WP:PSTS) to change this, please let people know with a thread topic and probably a RfC. These pages are longstanding and have been in place for years, with a lot of debate. Discussions should be substantive rather than cursory link to WP:MEDRS; that would seem more like wikilawyering. We also need to remember the old adage that the perfect is the enemy of the good. Not all of us are right next to world-class medical research libraries. Medical review journals tend to be locked up pretty tight behind paywalls. That doesn't mean that people are prohibited from contributing. This is not Scholarpedia; it is the encyclopedia anyone can edit and everyone knows it and the limitations which that involves. II | (t - c) 05:52, 9 March 2012 (UTC)

Just as a matter of the actual policy, let's quote it:

Wikipedia articles should be based on reliable, published secondary sources and, to a lesser extent, on tertiary sources. Secondary or tertiary sources are needed to establish the topic's notability and to avoid novel interpretations of primary sources, though primary sources are permitted if used carefully. Material based purely on primary sources should be avoided. All interpretive claims, analyses, or synthetic claims about primary sources must be referenced to a secondary source, rather than to the original analysis of the primary-source material by Wikipedia editors.

Ok, so there's a difference between "prohibited" and "generally unwise" but let me tell you that every time I've seen someone argue that "... says you can use primary sources" they have been trying to do something wrong. To me, it is like driving on the correct side of the road. There are times when you need to drive on the other side, but you don't base your driving style on it. If you find yourself getting into an argument over using primary sources to present some fact or cover some topic, then it is likely you are on the wrong side. Like WhatAmIDoing's favourite "challeged" aspect to sourcing, nobody is challenging Sandy's text at Klazomania and I'm quite sure she's more than capable of gleaning something good from inferior material. There is a spectrum of source qualities and few sources are absolutely prohibited. We don't encourage the use of primary research papers, per policy, just like we don't encourage the use of abstracts or newspapers or press releases. There are times when these source types are helpful, mostly where the topic isn't well documented. I agree with WhatAmIDoing's "fill in the blanks" point. For example, in the total body irradiation article, look at this patient information leaflet. It is full of useful information that I bet nobody would ever mention in a review or a chapter in a medical text. I wouldn't use it to list the indications or effectivenes for various conditions, but we can use it to fill in the blanks. For the big article topics like obesity, cancer, epilepsy one should be easily capable of writing all that needs to be said using secondary sources. Colin°^Talk 09:03, 9 March 2012 (UTC)

I believe that the patient information leaflet would be considered a secondary source for information about TBI. Secondary is not another way of spelling "high-quality".

I am happy to use such "low-quality" (properly, non-academic) sources to support very general or basic information. I think it's good for readers to find a variety of types of sources, so that people who don't have a scholarly bent can find sources that are appropriate to them. WhatamIdoing (talk) 16:05, 9 March 2012 (UTC)

Sorry if that was confusing. I wasn't saying the PI leaflet was a primary source, merely that it was one of the types of sources we wouldn't recommend folk base an article on, like abstracts, and press releases, etc. Sometimes, you have to make do with what you can get. And sometimes, an atypical source can be exactly what you need: it is very difficult to get academic sources that discuss "what the patient goes through". Colin°^Talk 19:25, 9 March 2012 (UTC)

WebMD as a reliable source

The current guidelines explicitly name WebMD, stating that WebMD and other sources "are usually acceptable sources in themselves". I'd like to bring this into question, and propose removing WebMD from the list or changing the wording here. Lately I've been delving into more depth in material on which I am fairly knowledgeable, and finding serious problems with WebMD's research. Let me give two examples:

The following page on WebMD, [2], says "One study has found regular use of products containing tea tree oil may cause gynecomastia", and then cites the following study: [3], which, if you read the source, provides insufficient evidence to support the claim made by WebMD. The way WebMD has cited the source suggests they merely read the headline of the article or possibly the abstract, rather than actually reading the article.

The following page on WebMD: [4] makes the claim "black tea has the highest caffeine content", without citing any source. This is a topic I have researched extensively, as I maintain a page on RateTea about the caffeine content of tea. In particular, the following sources: [5] and [6] draw this into question. Also, although it's less relevant because it's not really a medical topic, the page also contains information about tea that is sloppy, i.e. that Pu-erh is "Considered a black tea", and that Wuyi is "One variety of oolong" (it is a region that produces oolong and other teas, not usually considered a variety).

Furthermore, I have contacted WebMD about the caffeine point, and I did not receive any response, nor has it been corrected. These may be only two examples, but they're examples of topics that I have researched the most thoroughly and know the most about. I suspect that if I really delved into depth, I would be able to find more examples of how the quality of scholarship on this site is lacking and is thus unsuitable for citing as a reliable source.

My questions are: do you think WebMD's quality is lacking relative to the other sources given? If so, we could remove it from the list. Or do you think that the quality of all these sources are relatively similar? In this case, it might be better to change the wording to "may be acceptable..." rather than "are usually acceptable..." Cazort (talk) 16:23, 3 April 2012 (UTC)

I have seen at least two instances of poor analysis and conclusions published in Cochrane Review and it is considered one of the highest standard sources. WebMD I imagine given it's size and scope has lots of health-care professionals writing for it, so some mistakes are inevitable. From my use of WebMD, they seem fairly comprehensive and accurate. If you see a clear error in WebMD, find a different good source to add a differing viewpoint or possibly replace the WebMD with it if it is a clear error on WebMD's part. I think removing WebMD as a recommended source is like basically throwing the baby out with the bathwater and will create more problems than it solves. Others may have a differing viewpoint though.--Literaturegeek | T@1k? 16:48, 3 April 2012 (UTC)

Hmm, you're right that even the best sources have problems. My impression of WebMD is an overall impression, not just based on these specific examples. My impression of WebMD is that it tends to present material that fits with societal "common knowledge", sometimes perpetuating myths without questioning, and that it does not show enough evidence of critical research for me to be comfortable with it being mentioned in the guidelines as it is here. I personally have a more favorable impression of Mayo Clinic, but I don't feel confident with my impression, it just is an intuition, and I have no opinion on the other sites mentioned. "Throwing the baby out with the bathwater" is less of a concern for me in this case because I don't find much unique material on WebMD, i.e. material that can't be found in higher-quality sources. If we leave WebMD and the other sources, I'd be more comfortable if we weakened the language to "may be acceptable..." or something similar. Cazort (talk) 18:19, 3 April 2012 (UTC)

I pretty much agree with LG: all of these are "usually acceptable", by which we mean that they are "sometimes not". Like any source, if they disagree with the general run of sources, or with sources that are higher quality and/or more appropriate (e.g., a source entirely about tea-growing regions for your Wuyi example), then they should be avoided. WhatamIdoing (talk) 19:22, 3 April 2012 (UTC)

I rarely use WebMD. There are concerns of conflict of interest per [7] but it is not the only source with one. While there are better sources I would not argue for an outright ban. Doc James (talk · contribs · email) 01:53, 4 April 2012 (UTC)

Hmm. I also would not propose an outright ban yet, but after reading this article, I'm feeling stronger about changing how we mention WebMD, relative to, say, Mayo Clinic (I haven't found any overt misinformation on Mayo Clinic, whereas I've pretty consistently found misinformation on WebMD when I've delved deep enough to assess the truth of the material there). I've had an impression that Mayo was more reliable and it's seeming an increasingly poor choice to leave them in a list that implies they're roughly on the same level in terms of reliability. I have not had much experience with the other sites in the list so I don't really have any opinion on them at the moment. Regarding wording, I think there's a huge difference in connotation between saying something is "usually acceptable" or "sometimes acceptable". My personal inclination would be to consider WebMD to be "sometimes acceptable". For instance, I tend to be very skeptical of its recommendations about drugs too. For example, it has a ton of pages on Benzodiazepines which implies they are suitable for long-term or general use, like this page that provides it as an option for bipolar disorder: [8], yet there seems to be a growing medical consensus that these drugs are only suitable for short term use for acute anxiety. Mayo Clinic expresses this: [9] "Benzodiazepines are generally only used for relieving acute anxiety on a short-term basis". The potential conflict of interest, which I did not know about until reading that article, would make me more concerned particularly because of this sort of discrepancy. Cazort (talk) 14:15, 6 April 2012 (UTC)

"Unless the primary source itself directly makes such a claim"

The sentence "Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources" is wonderfully clear, and I've been wondering whether to incorporate it in WP:PSTS (on grounds that it's a logical consequence of existing policy, and hence applies to all articles). However, the final part of the sentence ("unless the primary source itself directly makes such a claim") doesn't make sense to me. If a source debunks, contradicts, or otherwise comments on the conclusions of a secondary source then it isn't a primary source. It's a secondary (or tertiary) source. What is it intended to mean, and how can it be rephrased to better express that meaning? Jakew (talk) 09:58, 24 April 2012 (UTC)

Agree we should get rid of it. --Doc James (talk · contribs · email) 10:37, 24 April 2012 (UTC)

(outdent) It means, if a primary source directly criticises a secondary source, for example, a primary source, may point out undeclared conflicts of interests or perceived flawed methodology used by reviewers or confounding variables etc. The primary source would basically be saying the review by such and such group of researchers is flawed because,,,,. The primary source would have to specifically mention a secondary source by name. It is NOT the same as using a primary source such as an individual study to debunk a review of studies. I don't think the line should be got rid of but perhaps it is possible to better word it?--Literaturegeek | T@1k? 23:55, 24 April 2012 (UTC)

But many of these comments are not WP:DUE. For example we have this report by the AHRQ [10] which was criticised by one of the leaders of the religious movement in question [11] --Doc James (talk · contribs · email) 06:10, 25 April 2012 (UTC)

Ah. As I see it, whether a source is primary or secondary depends on how it is used. If we cite a source for its criticism of another source, then we aren't using it as a primary source. We're using it as a secondary source. That's why I find it really confusing to see it described as a "primary" source. I think we've got used to thinking "primary source = anything in a primary research study, secondary source = anything in a review", which is often convenient, but it's an oversimplification. As Doc James points out, though, there are frequently undue weight issues associated with citing primary research studies.

How about rephrasing: Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources. Explicit criticism of secondary sources, however, may be included if appropriate, but be sure to assess due weight. Jakew (talk) 07:32, 25 April 2012 (UTC)

I don't agree with the suggested "Explicit criticism of secondary sources, however, may be included if appropriate". We really should try to mostly not discuss the sources in article text at all. Sometimes it is useful to explicitly mention research studies and meta analyses and even literature reviews, in text, but generally we should be in the business of describing facts, not the means by which folk discovered or write about those facts. So I don't want to encourage editors to argue/prove a point in front of the readers.

Note that the guideline text does not say the primary source debunks the text of the secondary source, or the methodology or even the moral character of the authors (that would, indeed, make it a secondary source on the review it was debunking. It says it explicitly debunks or contradicts the conclusions in the secondary source. I think what we're trying to avoid is something like this garbage: "A Cochrane review in 2006 concluded that drinking XXX fruit juice had no protective benefits on YYY cancer. However, recent research shows that XXX fruit juice contains aaa compound [link to research study on fruit juice composition], which is protective against cancer [link to research study injecting aaa compound into rats]. Neither primary research paper directly or explicitly contradicts the conclusions of the review. However, if somone did a long-term study of people who regularly drink XXX fruit juice and those who didn't, and found and stated that it did indeed appear to have protective benefits on YYY cancer.... That recent research could potentially be used to debunk the old review. It would be explicitly debunking/contradicting the conclusions, but doesn't even need to mention previous reviews, so doesn't become a secondary source. Colin°^Talk 12:04, 25 April 2012 (UTC)

When we talk about directly making the claim that it de-bunks older works, I think we are looking for a paper that says something like, "The received wisdom in the field (as seen in every textbook for the last 100 years) says that patients should not be permitted to eat anything for 24 hours after major surgery. However, as far as we can make out, this popular old idea is based on zero evidence, and we've actually done a proper randomized, controlled trial, which we outline here, and the data produced says that the old surgeons' ideas about low diet is a bunch of bunk."

Our approach to using them (the textbooks and the RCT paper) would then look something like "Keeping the patient on a low diet after major surgery has been widely recommended,^[1][2] but newer research suggests that it may be unnecessary.^[3]"

If, on the other hand, we had the same study, but the authors made no direct mention of how their study supersedes the older works, then we would probably ignore it (for now): "Keeping the patient on a low diet after major surgery has been widely recommended.^[1][2]" WhatamIdoing (talk) 13:51, 25 April 2012 (UTC)

I disagree. I don't think there is any requirement for the new source text to explicitly name the debunked sources or to imply them with a remark like "every textbook for the last 100 years". The "de-bunks older works" comment falls into the same trap of confusing source text/work with the facts it presents. Criticism of older "works" might happen in a field like history, say. But research in medicine would tend to overturn previous "facts" rather than "works". Also, there's nothing the research authors can say or not say that makes any difference to the WP:WEIGHT we give to their findings. So the argument that, depending on what they say, we could ignore them or not, doesn't work. The guideline we are discussing is preventing original research based on primary sources. It is a separate issue to consider whether that primary research paper's conclusions have enough weight to warrant mention. Colin°^Talk 15:22, 25 April 2012 (UTC)

I have to say, I'm not comfortable with citing primary sources in either situation.

In the case that the authors assert that their study is more significant than the sum of previous work, their assessment is unlikely to be wholly objective. They may well be correct, but it wouldn't be surprising if their view of the importance of their work was a little out of proportion. We're all prone to thinking our work is more important than it really is. I would generally think it best to wait until a secondary source cites their paper and judges the significance of their work from a less biased perspective.

In the case that they make no particular assertion, I'd be very concerned if editors were to make that assessment themselves. True, much of the time the decision will be rational and in accordance with principles of evidence-based medicine, but many controversial topics are subject to widespread POV pushing, and it would be best to avoid long-winded debates full of original research about the relative importance of various studies. Again, why not just wait until a secondary source becomes available? It's unlikely to be that urgent, after all... Jakew (talk) 09:35, 27 April 2012 (UTC)

I agree with those above who think that the phrase "Unless the primary source itself directly makes such a claim" is more problematical than helpful. I think there is a consensus for its removal, so I'm going to test that by removing it. I understand that it is possible that a better worded qualification might be preferable, so feel free to revert me and continue discussion here if I've misjudged the opinions expressed so far. --RexxS (talk) 15:32, 2 May 2012 (UTC)

MEDRS - too complex?

I think the guidelines has become too complicated for its own good, specifically about primary sources. For example this vital article http://news.harvard.edu/gazette/2006/12.07/11-dairy.html would be valid in most places but not here, this really concerns me and shows the flaws of Wikipedia and the urgent need for revision of the guidelines in general of any field. Helios solaris 16:11, 26 April 2012 (UTC)

It is complex because the problem is complex. The article you link is a newspaper/magazine type of article based on a talk someone gave in their lunch hour. Why do you think a serious encyclopaedia should base its articles on that kind of "I've got a radical new idea and have cherry-picked a bunch of studies that support it" kind of journalism. -- Colin°^Talk 16:36, 26 April 2012 (UTC)

Because if not it will fall behind. Helios solaris 17:33, 26 April 2012 (UTC)

I don't mind "falling behind" speculation and cherry-picking. The researcher in your source says that her hypothesis is unproven and not ready for prime time.

That said, this source is usable, for certain limited purposes. You could use it, for example, to support a claim that research is being conducted on the relationship between dairy consumption and hormone-sensitive cancers. You can't use it for a claim that any such link has been proven, or that readers should change their diets. WhatamIdoing (talk) 19:04, 26 April 2012 (UTC)

There is insufficient evidence of WP:DUE. We have much better sources discussing what research is ongoing. --Doc James (talk · contribs · email) 10:19, 27 April 2012 (UTC)

When you realise how hard it is to find secondary sources in the way that pleases Wikipedia and add it, even then it's suppressed, it becomes obvious that they are suspiciously superfluous. They exist only to subdue true evidence and strenghten the corporate establishment, just remember the tobacco industry do I need to say more. Critical thinking is clearly not welcomed here. --Helios solaris 16:25, 27 April 2012 (UTC)

There's nothing wrong with critical thinking, we just don't let it affect our edits. WP:NOR is long established policy for one very good reason: when "anyone can edit", some anonymous/pseudonymous editors will be incapable of critical thinking. If we allow it for you, we have to allow it for everyone, including those who can't think at all. Instead, we insist on wp:verifiability and secondary reliable sources. It works, though we have to keep explaining why we do it. LeadSongDog come howl! 19:34, 27 April 2012 (UTC)

I agree with Helios solaris. I witnessed in here how the rules were used in order to serve commercial interests. This is in particular against public interest, when it comes to adverse effects to the commercial products.

Important side effects of the absorbed radiation, of a dose that CT impose, were not accepted, because of bad interpretation of the rules. The side effects included the extent of the caused DNA damage, cataracts, circulatory problems, and cognitive impairments. --Nenpog (talk) 15:12, 1 July 2012 (UTC)

None of the above is remotely true. Nenpog didn't get his way in a content dispute[12][13][14], and now he is WP:FORUMSHOPPING with vague accusations of wrongdoing by anyone who dared to disagree with him. Previous forums where he tried to get his way and was shot down are: Talk:X-ray computed tomography, Wikipedia talk:WikiProject Medicine, Talk:Ionizing radiation, Wikipedia talk:No original research, Wikipedia:Dispute resolution noticeboard, User talk:Elen of the Roads, Wikipedia:Conflict of interest/Noticeboard, Wikipedia talk:Neutral point of view and now Wikipedia talk:Identifying reliable sources (medicine) . --Guy Macon (talk) 02:05, 2 July 2012 (UTC)

Please ignore what Guy Macon wrote. The Guy is following me around where ever I post, and is tailing my posts, in order to prevent neutral consideration of my point of view. Is that how Wikipedians expected to act? Please take a look at the discussions that he has pointed out, in order to see how the rules were misinterpreted in favor of commercial interests. --Nenpog (talk) 05:31, 2 July 2012 (UTC)

So it is your contention that you should be allowed to go to multiple forums and accuse others of "[breaking] the rules [...] in order to serve commercial interests",^{(The previous quote was edited to add brackets/ellipsis)} but the targets of your false accusations are not allowed to follow you around as you shift from forum to forum? --Guy Macon (talk) 05:59, 2 July 2012 (UTC)

The discussion here is about the MEDRS rules. Helios solaris made a point regarding commercial interests, and I agreed with him. You quote "breaking the rules" from where? No mentioning of that was made here. You followed me also in there, where no mentioning of commercial interests were made. Why do you think that everything is about you ("the targets of your false accusations"_{Guy Macon})? You agreed in here that I didn't made any specific accusation against anyone. Here is an idea, how about sticking to the topic of the discussion, instead of interrupting discussions with your personal issues. I will not converse with you or your proxies in here again about your off topic personal issues. --Nenpog (talk) 07:12, 2 July 2012 (UTC)

Very nice and fine point there Nenpog, you said "the rules were used" rather than "breaking the rules", so indeed Guy should not have used quote marks. Around here we call that wiki-lawyering, a device used to evade the actual point under discussion. It doesn't matter, your accusations are still false, including your "suspicion" of Guy personally. It completely baffles me why, if you are so right, you can't just find some higher-quality sources to support your view, but whatever, not just Guy Macon is watching your edits here and elsewhere. I would suggest you should stop shopping at this particular forum. Find better sources or drop the subject, OK? Franamax (talk) 08:43, 2 July 2012 (UTC)

I am beginning to think he is going for a new record. He only has a few forums left. He hasn't tried WP:SPI, Talk:Reliability of Wikipedia, Wikipedia:Arbitration/Requests or User talk:Jimbo Wales yet, and then there is always Wikipedia Review... --Guy Macon (talk) 12:08, 2 July 2012 (UTC)

Section tagging

Various templates (such as {{MEDCN}}, {{MEDRS}}) are available to tag reliable source problems inline. Is there one to flag up a whole section, i.e. some "medical source" version of {{Refimprove}}? If not, I think there should be. Peter coxhead (talk) 10:34, 1 May 2012 (UTC)

Sounds like a good idea. --Doc James (talk · contribs · email) 10:36, 1 May 2012 (UTC)

Well, if there isn't one already, how about something like my draft at User:Peter coxhead/Test/T1? As with {{refimprove}}, on which it is based, using the 1st parameter |section will replace "article" by "section". Peter coxhead (talk) 11:12, 1 May 2012 (UTC)

{{refimproveMED}}, though I don't know whether it has a section parameter. Nikkimaria (talk) 13:30, 1 May 2012 (UTC)

{{Medref}} (where did that refimprove come from and which do we prefer), but no section parameter. SandyGeorgia (Talk) 13:32, 1 May 2012 (UTC)

Neither of these have section parameters, although they could be added. {{RefimproveMED}} doesn't do what I want, because it says that the article is a health or medicine one, whereas I want a template to be added to e.g. a plant article that has a section on its supposed health benefits. {{Medref}}, too, suggests more to me that the whole article is a medical one; I'm not sure that WikiProject Medicine would really want to get involved in sorting out a few odd herbal medicine claims added to what is mainly a plant article.

But I don't want to create yet another template unnecessarily, so what do others think? Could {{Medref}} be worded so as to be a bit more appropriate to a non-medical article? I do think it's worth including this quotation "Biomedical information in articles must be based on reliable, third-party, published sources and accurately reflect current medical knowledge" especially in a warning meant for a non-medical article. Editors who add this kind of information to plant articles simply don't know about this requirement. Peter coxhead (talk) 14:23, 1 May 2012 (UTC)

Medref now has a section parameter: {{medref|date=May 2012|small=1}}. Cheers, — Bility (talk) 16:27, 1 May 2012 (UTC)

Medref should now be appropriate for use on non-medical articles, too. The wording settled on ("This article needs more medical references for verification.") and template doc I wrote ("This template is intended to be placed at the top of articles with medical or health content") should cover health sciences related content in an otherwise non medical article. Further tweaks can be made if necessary. It now has a reason parameter |reason=foo is bar for giving specific details. Thanks, --92.6.211.228 (talk) 18:15, 1 May 2012 (UTC)

Actually {{Medref}} doesn't have a section parameter. This line:

| text = This article '''needs more [[Wikipedia:Identifying reliable sources

needs article replaced by {{#if:{{{1|}}}|{{{1}}}|article}}

exactly as in {{refimprove}}, {{no footnotes}}, {{more footnotes}}, {{how}}, etc. In all of these, if you use {{TEMPLATE-NAME|section}} the message becomes "This section needs more ..." instead of "This article needs more ..." Peter coxhead (talk) 21:57, 1 May 2012 (UTC)

It does. Sort of. If you look at the testcases (down the bottom), you can see it's different to those templates in that the small/section version doesn't say "This article" or "This section". --92.6.211.228 (talk) 22:10, 1 May 2012 (UTC)

(edit conflict) It's not a parameter called section that I meant, but a parameter you can use for the small version in sections, which is what I thought you guys were talking about. Right now the template uses this first unnamed parameter as the rationale, so you don't have to explicitly use |reason=. We can change this, although if you want to use the small version in a section, it's a moot point since that line won't be displayed. This would also mean any existing templates using the first unnamed parameter for their rationales would need to be updated to use |reason=. — Bility (talk) 22:36, 1 May 2012 (UTC)

Added it. Hmm. It replaces the word article with the reason value when all params are used. That's fixable of course. Did you want it to replace |small? Allow both? --92.6.211.228 (talk) 22:27, 1 May 2012 (UTC)

I've added this test {{#ifeq:{{{1|}}}|section|section|article}} to {{medref}}. If, and only if, the first parameter is exactly the word "section", then the template will display "This section ..." instead of "This article ...". This change does not affect cases where the first parameter is a reason. I've used this in an article (Paris polyphylla#Medicinal uses, as of now) and it works fine. Peter coxhead (talk) 08:28, 2 May 2012 (UTC)

Actually I had to make another change to stop the word "section" appearing as a reason as well as forcing "This section ..." when |reason= was absent. I've tested all the cases and it should be ok now. Peter coxhead (talk) 09:18, 2 May 2012 (UTC)

I've also added information about {{Medref}} to the documentation for {{Refimprove}}, which is where I went in the first instance and failed to find the more specific template. Peter coxhead (talk) 09:35, 2 May 2012 (UTC)

Add to Multiple issues?

I think it might be worth adding it to {{Multiple issues}} in line with other standalone templates. Wording (examples) needs to be agreed on.

Something like:

It needs additional medical references for verification. or

It relies on references which may not be reliable sources for medical or health content. or

It includes attribution to sources which may not be reliable for medical content.

Thoughts? --92.6.211.228 (talk) 18:45, 1 May 2012 (UTC)

Yeah, I was planning on getting it set up along with expert-subject, which is also missing, since I've already added medref to the multiple issues template on a couple articles. EDIT: Actually expert-subject is in there, just called "expert". Anyway, I'll go add an edit request for medref. — Bility (talk) 20:52, 1 May 2012 (UTC)

Just an FYI, the request to add medref to multiple issues was denied. I removed it from the three instances of multiple issues that I was aware of. Cheers, — Bility (talk) 16:55, 29 May 2012 (UTC)

The guideline should be changed

"Basic advice", subsection "Assess evidence quality", paragraph 4 includes this statement: " "Assessing evidence quality" means that editors should determine the quality of the type of study. Editors should not perform a detailed academic peer review. Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions. "

Recently, this study was proposed as a reference for the article "Chemotherapy". Several editors (including, latterly, myself) were critical of the study and objected to the use of the study as a reference. See full details here.

WhatamIdoing quoted the above text from WP:MEDASSESS as justification for inclusion of Morgan's paper as a reference. In my opinion, the text fails to take into account those rare secondary sources that are severely biased or flawed, such as Morgan's paper.

I see no reason why editors shouldn't perform detailed peer reviews if they so choose. Moreover, I believe that such reviews can be helpful in determining suitability as references for Wikipedia. Axl ¤ [Talk] 19:41, 1 May 2012 (UTC)

Are there other reviews that are more recent that contradict this paper from 2004? We do say evidence from the last 3-5 years. --Doc James (talk · contribs · email) 20:28, 1 May 2012 (UTC)

I don't believe that there are any other reviews of this nature – because the whole premise of the review is flawed. Axl ¤ [Talk] 20:31, 1 May 2012 (UTC)

For information: the disputed text was added on 27 March 2010 by WhatAmIDoing. A talk page note at the time of the edit did not bring about any further discussion. However, the note refers to this earlier discussion and the whole section was added by Eubulides along with this discussion. -- Colin°^Talk 20:36, 1 May 2012 (UTC)

I think we should allow editors to exercise judgement in deciding whether a secondary source is actually suitable. There are loads of secondary sources that are of insufficient quality. In that sense, the "peer review" should assess the quality of the source. Disputes as to the suitability should be resolved by consensus. In this case, there was consensus that the source was not suitable. JFW | T@lk 21:46, 1 May 2012 (UTC)

Yes agree that consensus should be followed over the guideline. If the majority of editors do not see this ref as suitable than it should not be used. Does not necessitate a change in WP:MEDRS though. Many people still fell that WP:MEDRS says using primary sources is fine when high quality secondary sources are available which IMO is a more pressing issue.Doc James (talk · contribs · email) 21:54, 1 May 2012 (UTC)

As per User:JFW and User:Doc James above. Of course secondary sources are vulnerable to bias (and original studies can contain high-quality 'secondary source' material). Metanalyses and other systematic reviews are genuine 'studies' in their own right. Since WP:MEDRS needs to be widely comprehensible and has to safeguard against real-world dangers such as POV pushing and poorly informed editing, it can only provide rather broad guidance. So well-informed talk page discussions can be key. —MistyMorn (talk) 11:24, 2 May 2012 (UTC)

I share some of WhatAmIDoing's concerns (expressed elsewhere) that we have to come up with rules that work for the POV pusher and the undereducated. I'm one of the undereducated. My brain is much smaller than that of MastCell and Axl and WhatAmIDoing; I have no health training beyond a first aid badge when I was in the Scouts; I'm just someone's dad. Some of WP's rules are here purely because of the special situation with our authors: we don't really know who they are; we don't trust who they say they are; most of them are bonkers; some of them aren't geniuses. An editor who is an expert in the subject is likely to find some of these rules frustrating at times.

The problem with the Chemotherapy paper raises two issues. The first is that academic papers, whether primary research or reviews, are written for a certain learned audience, and not for general consumption. That PubMed has made them (and particularly their abstracts) so accessible, is wonderful but at the same time very dangerous. As the article talk page shows, the abstract is over-simplified to the point of being misleading. And it takes someone quite knowledgeable to discover what the analysis actually looked at and is capable of saying. Combine this with the possibility that the paper is flawed/biased and we have a problem. Fortunately, it is not a common problem, and is less of a problem than editors trying to pick/review primary research papers.

The second issue is the level of source we are using here. There's too much emphasis on review papers among WP:MED folk in my opinion. Yes, it is wonderful to be able to read all this material on your browser or PDF on the computer, and search it and collate it on your hard disk. I do so myself. But what about books, guys, books! Chemotherapy is a big subject. I just searched Amazon on it and turned up several suitable texts including ISBN 160831782X: "Handbook of Cancer Chemotherapy", edited by Roland T. Skeel, Samir Khleif, published 2011 (an earlier edition is actually used by the article for one point, but sadly the page number isn't given). This is 832 pages of wonderful readable and up-to-date-enough material. Approximately 300,000 words by a rough estimate. Now our Chemotherapy article is about 4,000 words. Which means you could condense that book 75x to achieve the current text. That's about 5 words from every page.

The chemotherapy article is crap. There are many medical articles on WP I'd recommend to a friend, but that one isn't. Not by a long way. Why are we wasting days discussing the merits or otherwise of some 8-year-old review paper. Visit your university library bookshelves, or order a book (second-hand) from Amazon/etc -- you can always sell it later.

WP:WEIGHT is underused in sourcing disputes. Rather than wikilawering over PSTS or disecting the journal paper or whatever, just find out "What do experts generally say, in published reliable sources, when writing about this subject". We are used to using WEIGHT to judge research findings: we read and use reviews/etc to find that. Similarly, we should use WEIGHT to judge analysis reports/reviews, especially if editors dispute them. What do our serious weighty oncology textbooks have to say on the subject? Rather than fall out with each other over such things, find some expert person/body who has already thought hard about it, published their work, and use that. WP:WEIGHT specifically warns against editors forming their own consensus about what issues/facts to include in an article, so I disagree with JFW/Doc James on that point -- however I suspect if you follow my advice, the review and its conclusions would be rejected anyway. Colin°^Talk 22:09, 1 May 2012 (UTC)

I feel that, in this instance, the issue could be resolved by proper usage of existing guidelines. Colin suggested above several alternatives to giving infinite weight to a single source that is seemingly flawed. I briefly proposed another on Talk:Chemotherapy that involves exercising judgment not on the quality of the source, but whether the wiki text correctly reflect the conclusions being drawn by Morgan et al. To wit, is the source saying what the article is saying? In my mind, the source was misused because it was made to support a statement that is too broad. Unless a more systemic problem with WP:MEDASSESS is identified, I do not support making drastic changes to guidelines serving a useful purpose. Wafflephile (talk) 23:30, 1 May 2012 (UTC)

I didn't intend to re-hash the debate about the (lack of) merit of Morgan's paper here. Rather I wish to consider the value of this statement from WP:MEDRS: " Editors should not perform a detailed academic peer review. Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions. "

Why shouldn't editors perform a detailed academic peer review, if they so wish?

The second sentence is potentially misleading. With careful reading, it implies that only the type of study needs to be high-quality. As long as the source is a review article, it doesn't matter how poor its quality. It is inevitable that some secondary sources are better quality than others. The second sentence implies that we should give them equal weight, regardless of their quality. Axl ¤ [Talk] 12:16, 2 May 2012 (UTC)

Most editors aren't qualified, or indeed competent, to perform a detailed academic peer review, and we've no mechanism in place to identify those who are competent and give their views more weight. Consequently, we've no way of allowing experts to reject papers that are truly dire without also allowing POV pushers and fringe theorists to reject papers that they dislike, on the grounds of nothing but their own (often ill-informed or irrational) opinion. It's a similar situation to allowing editors to directly insert OR into articles: an appealing option in some respects, but dangerous in others. Jakew (talk) 12:28, 2 May 2012 (UTC)

Does the fact that "most" editors aren't "qualified" constitute a justification for allowing lunacy to be perpetuated? If I see some kind of glaring flaw in a study or review, am I supposed to act like nothing's wrong? I always thought of Wiki articles as de facto reviews in some cases--balanced, dispassionate, factual surveys of whatever knowledge exists in a given area. Does the fact that my objection to a reviewer's statement that black is white has not yet resulted in a "comment" in a journal or an entry in a blog mean that, because I cannot quote an objection, that we have to pretend that black is white until we can document an alternative position? Here's an example: I discovered in http://www.ncbi.nlm.nih.gov/pubmed/3569020 that Burzynski, in 1987, theorized that one of his panacea molecules, #A10, interacts with DNA (which it does) to compete with carcinogens and thereby keep tumors from growing. The problem is that once tumors are formed, the carcinogen has nothing to do with it, and his molecules competing with carcinogens would have no conceivable bearing on the treatment of tumors (although, if correct, it is conceivable that they might be chemopreventive.) I have seen all kinds of criticism of Dr. B, but I have not seen anyone make that particular observation (maybe I missed it). If I were to be doing some kind of Wiki editing of his stuff (that is probably just as joyless as commenting on chemotherapy itself; I'm not even going to look), would I have to refrain from commenting on that? To me, it indicates an intellectual error that undermines the entire basis of what he thought he was doing. How could it be against Wikipedia policy not to point out the obvious? I don't think of Wikipedia as having to play the "emperor's new clothes" game.

Even the most prestigious institutions sometimes publish material that cannot be allowed to be unquestioned, and certainly, we all know that possessing even a Nobel Prize does not immunize the recipient to error or to lunacy. (Dare I suggest that it makes it more probable?) "Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions"--does that not strike you as insane? I wouldn't not mention it, but it is totally unconscionable to allow lunacy to be perpetuated. Despite what I just said, all other things being equal, I am going to give more weight to something out of a top-tier institution, but I'm going to examine each item on its own merit. Nobody gets a pass. Let me give another example of something I could not allow to pass, although it comes from a highly reputable institution: http://www.ncbi.nlm.nih.gov/pubmed/20043074, from MD Anderson. (Fortunately, that "study" did result in a fairly instant quotable counter-argument, such as at http://scepticsbook.com/2010/02/14/a-giant-leap-in-logic-from-a-piece-of-bad-science/, which also has the pdf of the original paper.) If there were no quotable counter-arguments, would Wikipedia have to go into the business of letting people believe that a solution containing no molecules of active ingredients is somehow capable of killing breast cancer cells?

I think we also need better definitions than "reputable" medical journals (what is disreputable, and to whom?), and "widely recognised" standard textbooks (widely recognized by whom), "disreputable" journals or "disreputable" fields or "respected" publisher. Maybe there is an explanation for those terms somewhere; my own feeling is like Justice Potter Stewart: "I know it when I see it." We do have to maintain the attitude the consensus might be wrong on something, as well as what I have found, that even the woo usually has some grain of truth to it.

GeorgeButel (talk) 05:16, 3 May 2012 (UTC)GeorgeButel

Because it isn't necessary. Because it is the same kind of argument that a POV pusher will use and you will fail to convince him just like he will fail to convince you. If you disagree with the conclusions of a source, then so will other people. If lots of people agree with your view, then it shouldn't be hard to prove that using a source-based argument rather trying to explain the merits or deficiencies of the source/study. If you can't find other experts writing about this issue in a way that agrees with you, then either the article should contain the alternative viewpoint, or perhaps the whole issue doesn't deserve the weight editors think it does.

Selecting sources, in general, is only the start. Then you need to work out what the article is going to say. Sourcing debates, without context, are somewhat pointless. The discussion didn't focus on the real questions facing article writing: What exact facts or other point would someone wish to draw from that source, and how would we write it? Is that the best source we can find for those facts (we already know it is 8 years old, which counts against it)? What do other high-quality sources have to say about that fact? Even if the fact isn't disputed, what weight is given to it in high-quality sources that discuss the general topic surrounding the fact, or the article itself? As a quick-and-dirty exercise, I searched the textbook I linked above, using Amazon, to see if it cites that paper or its authors. It doesn't. If a 300,000-word treatise on the article topic fails to use that source, why should we in our 4,000-word piece? Colin°^Talk 12:46, 2 May 2012 (UTC)

(edit conflict)

The thing is not that expert editors shouldn't conduct such peer review. Indeed they are free to do so as a reviewer in the real academic publishing world, not on Wikipedia. Of course having done so, they should be cautious of a new COI on Wikipedia. That paper they reviewed off-wiki doesn't become any more (or less) reliable or relevant on-wiki. Leave its selection up to other editors. So perhaps the sentence should be: " Wikipedia is not the place to perform a detailed academic peer review. Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions. " LeadSongDog come howl! 13:06, 2 May 2012 (UTC)

That's a definite improvement. We should never try to dictate what editors can do outside of WP. Jakew (talk) 15:29, 2 May 2012 (UTC)

In general I find that incidents like this are best dealt with by finding sources that criticize the original source and either juxtaposing the two in text, or on the basis of the validity of the criticism, not opting to include the study (for instance, specific to Morgan, Ward & Burton, there is a post by Orac on scienceblogs as well as a letter to the editor). In this case, if I cited MW&B at all I'd phrase it as a minority point of view ("In 2004 WM&B said...") with an accompanying reference to the LTTE ("Mileshkin, Rischin, Prince & Zalcberg said in a LTTE that...") It's pretty rare that an article genuinely bucking the mainstream POV is unanswered. In my experience, situations like these that are unanswerable are the minority - actual experts are not stupid, they usually rebut the lunatic fringe stuff.

Regards this section of MEDRS, I wouldn't want to throw the baby out with the bathwater - I've run into far more issues where I've referred to that line/idea as a way of keeping in mainstream views ("You don't get to say that review article can't be included because some of the funding came from Pfizer"). It might be worth refering to WP:FRINGE or WP:UNDUE, since that's the nub of the issue in this case. The idea that chemotherapy is worthless is a fringe idea, most doctors clearly think there is merit to it. WLU (t) (c) Wikipedia's rules:^simple/_complex 17:35, 2 May 2012 (UTC)

I don't understand WhatamIdoing's point (if I'm following correctly); sourcing is always a matter for consensus and discussion, and in any realm of editing (not just medical), editors can decide via consensus that a source is not appropriate. This guideline can't be excluding something that is general; we can always discuss, review, and come to consensus to not use a bad source. I must be missing something, because this discussion makes no sense to me. There are plenty of sources about Tourette's, for example, that even though high quality, are just wrong and shouldn't be used. For example, the New England Journal of Medicine published a review about 10 years ago that had the wrong basic definition of TS; editors can review that, see that, and decide not to use a source that is just wrong. Why are we saying editors can't come to consensus to not use a bad source on medical articles, when we can do that anywhere else? SandyGeorgia (Talk) 18:22, 2 May 2012 (UTC)

The guideline refers to "study". I think "sourcing", "study" and "facts" are getting mixed up here. Discussing sourcing without context can only deal with generalities like the type of source or type of publication. Some of our best writers are experts in the subject. Some of our worst POV pushers are experts in the subject. Both can produce lengthy talk page discussions that don't actually focus on proposed article text. What facts/points do we want to say? Are current books and reviews on the topic mentioning it? If not, then let's move on. Colin°^Talk 20:09, 2 May 2012 (UTC)

Sandy, the first objection to the source on the talk page was that if patients knew the facts about (cytotoxic) chemotherapy's limited contribution to cancer treatment (in most, but not all, solid tumors, surgery is far more important), then some of them would refuse treatment. Do you think that making patients make the "right" choice in the real world is a legitimate reason for entirely removing all information about efficacy from the article? I don't, but that's what's happened.

The TS comparison is inapt, because in that case, you had dozens of sources that gave a different definition. You could compare it to other sources and choose the many rather than the one. Here, we have just one source (that anyone knows of) that provides the overall statistic. (The point behind the paper was to calculate the overall benefit of all cytotoxic chemotherapy regimens for all patients in all stages of all invasive solid tumors.) It's not like this paper says 2% of survival the five-year mark, and another paper says 5%; this is the only paper that seems to have tried this ambitious approach.

There are good reasons to be careful about how it is presented: it applies only to invasive solid tumors, which means no leukemias (and so understates the benefit) and very little skin cancer (and so overstates the benefit). It considers only cytotoxic drugs, which excludes most of the newer stuff (hormones, monoclonal antibodies, radioactive drugs, etc.). It is looking for a population-wide number, so it includes even people who didn't need or didn't take chemotherapy (this is the difference between "pounds of beef eaten per American" and "pounds of beef eaten per beef-eating American").

These qualifications seem to form the basis of the technical criticisms: they chose to exclude hematological malignancies and non-invasive cancers (both very typical exclusions, by the way), and I would have made a different choice (as a point of fact, I personally would have made a different choice), so it's an unreliable source. They chose to consider only cytotoxic drugs, and I would have made a different choice (actually, I wouldn't have), so it's an unreliable source. They chose to consider all patients rather than some patients, and I would have made a different choice (I would have, too), so it's an unreliable source. NB that every complaint here is not "they got these facts wrong", but "they made different choices than I would have made".

Given that patients often believe chemotherapy is absolutely necessary, sometimes to the point of seeking it out despite multiple oncologists telling them that it is not appropriate, I think it sad that we now have an article that doesn't even mention undisputed facts about efficacy, such as that chemotherapy is hugely important to some cancers (e.g., testicular cancer), but provides very little benefit for others (e.g., lung cancer), even though not one of the people who dislikes this particular source believes that chemotherapy provides identical benefits for all forms of cancer. WhatamIdoing (talk) 22:16, 2 May 2012 (UTC)

The problem with "editors can decide via consensus that a source is not appropriate" is that sooner or later, you'll find yourself on the Abortion talk page where a group of editors have decided by consensus that a secondary, well-conducted review by the Guttmacher Institute is not appropriate "because the Institute is biased towards pro-choice". Or if not that page, then another where a group of SPAs have established their walled garden. I'm sorry, but we need MEDRS as a bulwark against POV-pushers, who would love the opportunity to disallow any reliable secondary source that didn't align with their own views. --RexxS (talk) 23:16, 2 May 2012 (UTC)

(edit conflict) @WhatamIdoing: I think we solve that problem by ensuring that our articles on testicular cancer, non-small-cell lung cancer, etc. each accurately reflect the role and efficacy of chemotherapy in those diseases. The parent article on chemotherapy should make clear that the efficacy of chemotherapy varies substantially by diagnosis and disease stage, and the details should be handled on a disease-by-disease basis. MastCell ^Talk 23:19, 2 May 2012 (UTC)

That would be fine with me—I have repeatedly said that the 2% statistic is unimportant—but you actually removed every single sentence in the article that "ma[d]e clear that the efficacy of chemotherapy varies substantially by diagnosis and disease". WhatamIdoing (talk) 16:33, 4 May 2012 (UTC)

RexxS, WP:CONSENSUS is a policy. In general, it trumps WP:MEDRS. While there may be a cabal of SPAs who are trying to subvert an article, WP:MEDRS cannot be used to overrule them. Such a cabal can only be suppressed by drawing on a wider consensus with more good-faith editors. Axl ¤ [Talk] 23:45, 2 May 2012 (UTC)

We already have a safeguard against that. Established consensus, in the form of policies and guidelines, trumps local consensus. Jakew (talk) 07:17, 3 May 2012 (UTC)

RexxS, welcome to the slippery slope that is all of Wikipedia (POV pushers gang up on good faith editors on talk pages-- try editing Venezuelan articles for a year or two). If you want to use MEDRS to trump CONSENSUS, pretty soon a carefully crafted and well applied guideline will fall into disfavor because it's used as a bludgeon and held to a standard higher than the policies it guides. AKA, shooting ourselves in the feet. MEDRS is a guideline, it enjoys consensus as such, it doesn't trump policy, and if folks start using it inappropriately, they're going to ruin a good thing. I guess I'm not understanding this argument, because whether one review is better, more recent, whatever is something that we can always discuss on talk and those discussions guide our decisions about which reviews to use. SandyGeorgia (Talk) 03:02, 3 May 2012 (UTC)

I'm sorry you thought my contribution was "really stupid", per your edit summary, Sandy. I can assure you that I have the scars from defending NPOV on numerous articles over several years, and we don't need to engage in a pissing contest over who does the most defending – but thank you for the belated welcome anyway. Standing up for NPOV is not a card game, and there are no trumps, but you must recognise that the project-wide consensus enjoyed by MEDRS is not to be ignored lightly by a small group arguing a WP:LOCALCONSENSUS: "Consensus among a limited group of editors, at one place and time, cannot override community consensus on a wider scale" (and that's policy). If every good-faith editor decides it's better not to argue the merits of MEDRS for fear of it falling into disfavour, then we might as well hand over the encyclopaedia to the POV-pushers right now. The practices documented at MEDRS are exactly what we have to convince other editors to use when building a consensus for the neutral point of view, because they are inimical to the methods used to push POV. The problem is you want it both ways: you want to be able to use your expertise to discard a poor secondary source, while expecting others to defend good secondary sources that have been labelled "poor" by a self-proclaimed expert. Have the lessons of Essjay still not been learned here? --RexxS (talk) 12:40, 3 May 2012 (UTC)

I'm disappointed by WhatamIdoing's response as she's falling into the same trap: using her expert knowledge to try to convince other editors, on a talk page, to include/exclude certain facts from the article. And to focus on the good/bad points in a study (or present counter-arguments against other editor's points) rather than discuss what we want to actually say and then present sources that support it. This is something Eubilides was great at: sourced-based discussion of actual or proposed article text.

WP:WEIGHT is policy. It says

An article should not give undue weight to any aspects of the subject but should strive to treat each aspect with a weight appropriate to its significance to the subject....Keep in mind that, in determining proper weight, we consider a viewpoint's prevalence in reliable sources, not its prevalence among Wikipedia editors or the general public... Also, if you are able to prove something that few or none currently believe, Wikipedia is not the place to present such a proof. Once it has been presented and discussed in reliable sources, it may be appropriately included.

I agree that it is very important how the facts in this case are presented, and that the disputed paper's abstract does an appalling job of that. The exclusions (particularly of modern drugs) are problematic and it will be hard to work out what should be said that is actually relevant to a modern reader in a general article on chemotherapy. But hey, I've got an idea. What do modern texts on chemotherapy say on the subject? It keeps coming back to this. Why on earth are our editors arguing over the merits of an 8-year-old study as a source for an article on chemotherapy in 2012. And since the benefits of chemotherapy seem to depend completely on what cancer is involved, the approach taken by the book I linked (where each type of cancer gets its own chapter) seems to be ideal for writing targeted text on each indication or otherwise for chemotherapy. Colin°^Talk 08:01, 3 May 2012 (UTC)

Agree if there are better more recent secondary sources we should be using them. Our guideline recommends against sources older than 5 years when more recent sources are available which I am sure is the case for chemo.--Doc James (talk · contribs · email) 15:05, 3 May 2012 (UTC)

(edit conflict) Agree that Colin's book suggestion could be a good starting point in this particular case (though as a general bibliographic point, perhaps it's worth pointing out that the quality of scientific medical books can vary from standard reference texts, like this, to pot boilers with little or no peer review). Ideally, one would like to have a widely respected reference textbook which is periodically updated. Even so, working up a single, high-quality page on such an extensive topic as chemotherapy remains a major task. This is a genuine issue, imo, because in practice generic pages such as Chemotherapy can be widely consulted and regard matters of considerable real-world interest. 2c only, —MistyMorn (talk) 11:32, 4 May 2012 (UTC)

So what exactly should be changed and how?

Editors should not perform a detailed academic peer review.
Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions.

Sentence (1) can be left out, it has no practical consequences. Sentence (2) - if a study has inclusion criteria, methodology or factual errors leading to generally implausible results than apply WP:FRINGE.

Sometimes the results are not even generally faulty, but study design or data (in)availability make them useless for encylcopedic purposes. As an example, endometriosis has a Cochrane Review of TCM as a reference. This kind of source is in my opinion fairly useless - even if TCM may somehow help endometriosis the design of the study is not good enough for anything except "a call for further research" which does not belong there. Richiez (talk) 11:22, 4 May 2012 (UTC)

I'm not sure that removing the first sentence would be a good idea. As I interpret it, the current text states that editors shouldn't perform a detailed peer review and then lists as examples some personal objections that are invalid. If the first sentence were removed, then the resulting text would read as though these four types of objections were an exhaustive list of the only invalid grounds for exclusion. But they aren't: for example, editors shouldn't exclude papers on grounds that they disagree with methodology (except the general type of study), assumptions, statistical analysis, ethical problems, etc. This follows naturally from WP:NOR, so it isn't strictly necessary to spell it out, but it's certainly useful to make it clear. Jakew (talk) 11:46, 4 May 2012 (UTC)

Oppose Richiez change suggestion. We don't want editors cherry-picking studies, which means we don't want them filling talk pages with their analysis of a study's failures. Full stop. It doesn't matter if the study is primary research or secondary analsysis of existing research. We build our articles on the work of expert writing, not by choosing which studies we think are good/relevant. You cannot apply WP:FRINGE based on your own view of what is fringe or not or by deciding that since you think the study is flawed, then it must be a fringe conclusion. The "Assess evidence quality" section is more background information than a primary guide to what to include in an encyclopaedia article. Have a look at a NICE guidleline like this one. The difference between the "NICE guideline" and the "Full guideline" is that the latter contains all the evidence for their guidance and how they went about measuring and collecting this evidence. It is a job for experts. It is a difficult job. It is not our job. Colin°^Talk 12:41, 4 May 2012 (UTC)

If we are disallowed to judge WP:FRINGE than we can either stop altogether or have a very short list of allowed sources such es Encyclopedia Britanica - and of course we can expand this to all of wikipedia not just medical articles. Richiez (talk) 12:54, 4 May 2012 (UTC)

We identify fringe theories/conclusions by the fact that (a) most reliable sources tell us they are wrong/fringe or (b) most reliable sources ignore the theory/conclusion -- in other words it has no weight to warrant mention in our articles. We don't identify them based on our own understanding. I dare say parts of the US think evolution is a fringe idea... Colin°^Talk 18:20, 4 May 2012 (UTC)

I'm with Colin. We don't write a guideline around primary studies because we typically shouldn't be using them anyway-- we use secondary sources and reviews, and we do come to consensus about which reviews are the highest quality. All of this discussion seems to have resulted because folks want to use and do original research with primary sources. SandyGeorgia (Talk) 14:08, 4 May 2012 (UTC)

I did not follow the chemotherapy discussion closely but it was my impression that it was a secondary source in question with results that happened to fall into WP:FRINGE category. Richiez (talk) 16:07, 4 May 2012 (UTC)

It's not really FRINGE. It's just a broader focus than other papers have taken, sort of like "how much has society benefitted frim vaccines" rather than the far more common "how much has society benefited from the polio vaccine". WhatamIdoing (talk) 16:33, 4 May 2012 (UTC)

It appears to be very much against mainstream in some respects. In breast cancer we try to differentiate by age, receptor status, markers or more recently gene activation patterns, response to neoadjuvant therapy and I think everyone is aware that the benefit of chemotherapy is likely to be dramatically different comparing a 40 year old or 85 year old women. It was my impression that the mentioned paper did the exact opposite of this approach by using unusually broad undistinguished patient population.

Now hypothetically assume there is a review that meets all formal criteria of MEDRS and is so fringe that "mainstream" does not really notice, let alone review it. With many thousands (somewhere I recall 33K academic journals?) this might happen from time to time. Applying [WP:FRINGE]] seems obvious? We have to use common sense. -- Richiez (talk) 21:10, 4 May 2012 (UTC)

Whether you regard it as fringe or something of no weight doesn't matter much. The issue for this guideline is that this must be judged with reference to our sources, not on the basis of our own opinions. This is the issue I had with "if a study has inclusion criteria, methodology or factual errors leading to generally implausible results than apply WP:FRINGE". Colin°^Talk 21:20, 4 May 2012 (UTC)

Why exactly should WP:MEDRS be exempt from the fringe rules? The essence of fringe is that someone, preferably by consensus regards some material as such. Richiez (talk) 21:39, 4 May 2012 (UTC)

You're not getting the point, Richiez. WP:FRINGE is a valid guideline but if you read it you'll discover that determining what is fringe or not is done with reference to published work, not by WP editors using their own brains to discredit some theory/conclusion. My second point is that this particular point on Chemotherapy doesn't have to be as extremely wrong as a fringe idea in order for us to give it no space. If none of our contemporary sources choose to comment on chemotherapy in the way that 8-year-old paper did, then neither should we. No matter what you or WhatAmIDoing personally think of the study's merits or flaws. Colin°^Talk 09:29, 5 May 2012 (UTC)

I support Richiez's recommendation.

" We don't write a guideline around primary studies because we typically shouldn't be using them anyway. "
— SandyGeorgia

That's a self-referential statement. We shouldn't use primary studies because our guideline doesn't recommend them? Anyway, this discussion is not about primary sources.

" All of this discussion seems to have resulted because folks want to use and do original research with primary sources. "
— SandyGeorgia

That is entirely false. No-one has recommended the use of primary sources, nor is anyone recommending original material in an article.

WP:FRINGE includes the statement "We use the term fringe theory in a very broad sense to describe ideas that depart significantly from the prevailing or mainstream view in its particular field." Morgan's paper does indeed depart from the mainstream view, thus it fits this definition of "fringe". Axl ¤ [Talk] 22:12, 4 May 2012 (UTC)

Axl: see my comment above. If his paper "does indeed depart from the mainstream view" then this should be evidence in the literature (either explicitly or because it is ignored) and therefore it is unnecessary for WP editors to attempt to prove this on a talk page with their own skill. When editors try to push other editors into accepting a viewpoint, without reference to contemporary high-quality sources, then that is POV pushing. Just because it is mainstream POV pushing, doesn't stop it from being something to discourage. Colin°^Talk 09:29, 5 May 2012 (UTC)

If you accept "beeing ingored" as evidence in literature than this would appear as reasonable - in theory. When you try to define this properly I expect it will turn out more difficult than a simple apply common sense rule. Eg how long after publication do you want to wait as evidence that something is ignored? We would be forced to publish something and wait 3-5 years for evidence it is ignored (and hope there is not a citation cartel at work)? Richiez (talk) 12:20, 5 May 2012 (UTC)

I've seen this problem before. The disputants will not agree on what is "mainstream" - one side says it's is; the other side insists it's not. If we let editors define mainstream, we never reach consensus. Where the editor's expertise comes in is to point others to how prevalent a view is in the quality literature (which is how we should be defining "mainstream"). If you're lucky, you find one review published in an in-house magazine and a blog, and the contradictory review published in the Lancet and the BMJ. Unfortunately it's often not so clear-cut. --RexxS (talk) 02:22, 5 May 2012 (UTC)

I've found taking a 'vote count' from a sample of reviews on PubMed to be a reasonable solution to this problem. It has its own problems, of course, but to get a rough measure of mainstream thinking it works quite well, and is obviously less susceptible to bias than relying on an editor's expertise. Jakew (talk) 07:21, 5 May 2012 (UTC)

sure that does happen, but I think there are many cases left where there is no doubt about what is not mainstream. Richiez (talk) 12:24, 5 May 2012 (UTC)

Observational studies (and CAM)

Although this is written specifically in the context of CAM (not about it...), I suspect we could use this principle in this guideline:

"Although observational studies cannot provide definitive evidence of safety, efficacy, or effectiveness, they can: 1) provide information on “real world” use and practice; 2) detect signals about the benefits and risks of complementary therapies use in the general population; 3) help formulate hypotheses to be tested in subsequent experiments; 4) provide part of the community-level data needed to design more informative pragmatic clinical trials; and 5) inform clinical practice."

Source: Observational Studies and Secondary Data Analyses To Assess Outcomes in Complementary and Integrative Health Care. June 25, 2012, Richard Nahin, Ph.D., M.P.H., Senior Advisor for Scientific Coordination and Outreach, National Center for Complementary and Alternative Medicine.

This is a RS from the NIH.

It clearly states that we wouldn't be able to use observational studies as RS to make statements of fact about "safety, efficacy, or effectiveness". This actually applies to far more than just CAM, and is very relevant for this guideline. How can this be incorporated here? -- Brangifer (talk) 01:14, 1 July 2012 (UTC)

Any individual study is a primary source, including observational studies, and this guideline strongly urges the use of secondary sources, such as research reviews and meta-analyses. So in a way it's already covered. But we could add it to this sentence: "Roughly in descending order of quality, lower-quality evidence in medical research comes from individual RCTs; other controlled studies; quasi-experimental studies; non-experimental studies such as comparative, correlation, and case control studies; and non-evidence-based expert opinion or clinical experience. " Perhaps after "other controlled studies." TimidGuy (talk) 19:49, 1 July 2012 (UTC)

That sounds good. It provides good information for those who don't understand such things. -- Brangifer (talk) 04:28, 2 July 2012 (UTC)

Thanks. Have added. TimidGuy (talk) 10:41, 2 July 2012 (UTC)

Mention the NCBI Bookshelf?

Should the NCBI Bookshelf be explicitly mentioned in the books section? Types: Report (792) Book (136) Documentation (32) Collection (23) Database (23) Similarly GeneReviews. RDBrown (talk) 23:09, 3 July 2012 (UTC)

That looks like a good resource for high-quality research reviews. I'd vote yes. TimidGuy (talk) 10:16, 4 July 2012 (UTC)

Do/should MEDRS and WP:V apply to content on the Wikipedia Reference Desk?

Wikipedia talk:Reference desk#How should we treat unsupported health-related content? --Anthonyhcole (talk) 06:40, 17 September 2012 (UTC)

If the reference desk is going to be useful yes. Doc James (talk · contribs · email) (if I write on your page reply on mine) 08:05, 17 September 2012 (UTC)

Disputed area of application

See Wikipedia:Articles for deletion/Biology and political orientation (2nd Nomination) where some participants have argued that studies in interdisciplinary journals etc. fail MEDRS. The other side of that sub-debate argues that the article in not providing any medical advice, so MEDRS isn't relevant. Tijfo098 (talk) 08:53, 26 September 2012 (UTC)

Does this fall under MEDRS?

Hi! I noticed this edit about a study regarding effectiveness. It seems a little promotional. Does this fall under MEDRS? WhisperToMe (talk) 20:21, 1 October 2012 (UTC)

Not really. Cost-effectiveness and access to care are not "biomedical information". WhatamIdoing (talk) 20:24, 1 October 2012 (UTC)

Ah, ok. Thanks WhisperToMe (talk) 21:07, 1 October 2012 (UTC)

I beg to differ with WAID on this one. It's true that cost-effectiveness is a form of economic analysis. But when talking about the cost-effectiveness of a medical intervention, the effectiveness part (ie the denominator of a cost-effectiveness ratio) implicitly incorporates health-status outcomes such as quality of life and survival; for example, a commonly used outcome measure is quality-adjusted life years (QALY) [15], where the cost-effectiveness ratio is expressed as $/QALY etc, allowing economic comparisons of different treatments/interventions to be made using incremental cost-effectiveness ratios. So I think there's a strong case for applying MEDRS for cost-effectiveness studies of medical or health interventions (especially when multiple primary studies are available).

Turning to the present case, the citation does indeed look highly promotional. The author of the piece seems to be Arthur Laffer, who is commenting a study "conducted by economist Arthur Laffer", which was sponsored by (wait for it) the company that performed best: Kool Smiles. Actually, to my layman's eyes the outcome measures don't even seem to regard cost-effectiveness (as claimed in the title), since the number of procedures cannot be considered a measure of effectiveness. The original report [16] appears to be published by Laffer Associates, scarcely a major peer-review journal. Ugh... Imo, this shouldn't satisfy RS, let alone MEDRS.

Adding: Would WT:MED perhaps be a more suitable venue for this discussion? Present example aside, the wider question about cost-effectiveness studies and MEDRS seems to me to be of relevance to the MED Project.

—MistyMorn (talk) 22:22, 1 October 2012 (UTC)

Nevertheless, cost effectiveness of a drug and pharmacological effectiveness are not the same thing. The pharmacological effectiveness addresses only whether the salutary effect of the drug can be seen against placebo or doing nothing, in real-world clinical practice. If patients don't like a drug and won't take it, that impacts effectiveness. If a drug costs a lot, that also affects effectiveness, but often less strongly (since many patients take drugs they don't pay for out of pocket, and even expensive drugs sell well if patients want them badly-- i.e. a drug that makes eyelashes longer, I kid you not). Cost-effectiveness can be measured by many metrics and QALYs per $ is only one of them. Ultimately, cost-effectiveness of drugs involves values and is as difficult to measure as cost effectiveness of transportation. Is a sports car as cost effective as a minivan? That all depends on how you define "quality," which is subjective. All this makes medicine look very much like engineering, although many a politician and government agency refuses to recognize that. Should your government buy you a new sports stadium, or instead a new bridge? Engineering can tell you how much these will cost, but not how you should spend your tax dollars. Medical evidence plays the role of engineering data, here; it's helpful but not the last word in what you should do.

Lastly I should note that all this is complicated by colloquial English having quite specialized uses in pharmacology. The FDA is charged to make sure that drugs are "safe and effective" but that does NOT mean the FDA actually looks at a drug's "effectiveness," in this restricted sense. Although it makes the head hurt, the FDA regards a "legally effective" drug as one that has pharmacological efficacy (in scientific clinical trials), not necessarily one that will be effective (in any sense), in real-world clincal office use.See page 4. The FDA actually looks only at safety and efficacy, the last referring to whether the shows any effect is the purest and most well-controlled trial, for which it only need show up statistically better than placebo. Whether this difference is clinically significant, or indeed will translate into a new drug that is "pharmacologically effective," or even "clinically effective" or "socially effective," is not the FDA's problem or business. Quite often the answer is "no," and yet these last properties of a drug are things that realistically must ultimately be traded off against safety issues (think, for example, Vioxx). If the FDA doesn't do that, the market and the tort courts will. And cost-effectiveness, even for proven-effective drugs, is yet something else again, and depends on who's buying. Helpfully, S B H arris 23:33, 1 October 2012 (UTC)

Thank you for commenting helpfully. Regarding the MED Project, irrespective of one's own personal views, I think it's fair to say that cost-effectiveness analysis is widely recognized as being of health relevance [17] in the real world, where financial resources are finite (and vary vastly from country to country). So it seems to me that, yes, MEDRS should apply. —MistyMorn (talk) 23:50, 1 October 2012 (UTC)

It may be, but that's not the question addressed above. The question was whether this item about economic cost effectiveness in dentistry (state-funded Kool Smiles vs. something else or nothing) was something addressable by MEDRS criteria, and the answer was that it was not, since MEDRS was designed to help in biomedical efficacy decisions ("biomedical information"), not social policy and healthcare delivery decisions (health care policy information). At which point you said that the "effectiveness" part of "cost effectiveness" implied the kind of "effectiveness" that MEDRS was designed to address. And the answer is "not nessarily," and probably not, for the reasons given above. MEDRS was designed to look at biomedical efficacy, which is an easy question compared with social and medical cost-effectiveness, which require input from a number of social values systems, and are subject to a lot of COI for that reason. All that problem of source and values go into regular WP:IRS where we deal with political problems.

The reason WP:MEDRS exists at all outside WP:IRS is that for the narrow questions asked by biomedical studies, there are algorithmic answers available as to what consitutes grades of evidence for various statements. As soon as you generalize to the kind of quesions WHO deals with, there no longer are. It's hard enough (for example) to figure out what the impact of steel crowns vs. fillings have in children's primary teeth in a very narrowly defined population, without having to deal (on top of that) with what happens when you have a corporation that does a lot of crowns for medicaid kids for medicaid dollars, being measured against non-medicaid dentists that do a different kind of care for kids whose parents have more money, but want to be paid with insurance or cash, which means the impoverished child whose family faces that, would often get no care at all. Steel crowned deciduous teeth eventually are replaced by adult teeth, and does the child suffer in the meantime? Is the tooth fairy outraged? Again, this is not really a biomedical question but a social one, much like the question of whether it is acceptable to have transportation of different quality for poor people vs. middle class or rich ones. A transportation engineer or transportation engineering study cannot answer that. Science cannot answer that. And even if the poor must eat hamburger rather than prime rib, what does MEDRS have to say about it? Not much. S B H arris 00:56, 2 October 2012 (UTC)

Since the broader question is clearly relevant to the Medicine WikiProject, I have started a thread at WT:MED. —MistyMorn (talk) 01:11, 2 October 2012 (UTC)

Cost-effectiveness

Following on from a discussion on WT:MED:

For example, two recent MEDRS available for implantable cardioverter-defibrillators:
- there's an updated systematic review conducted in connection with NICE [18]
- and a viewpoint-type article (arguably) presented and indexed as a review [19].
—MistyMorn (talk) 23:25, 2 October 2012 (UTC)

Unsurprisingly perhaps, our Implantable cardioverter defibrillator page currently doesn't specifically mention cost-effectiveness, but does approach the matter in a broader way, in the last paragraph of the Clinical trials section, based on a primary source (a multicenter study) [20] and an editorial comment (indexed by NLM as a review) [21] which does communicate the issues in an intelligible format. I'm wondering how all this stacks up and what the best editorial approaches might be in circumstances such as this .

—MistyMorn (talk) 12:19, 3 October 2012 (UTC)

Bluerasberry (talk · contribs) is Wikipedian-in-residence at Consumers Union, a non-profit organization, publisher of Consumer Reports, whose mission is to "test products, inform the public, and protect consumers." I would like to hear input from the consumers' union on communicating cost-effectiveness. I assume they have expertise and years of experience in communicating this kind of information to the layman. (Personally, I'd like the consumers' union to simply write good wikipedia articles. That would appear to me to be very much in line with their mission.)

Bluerasberry, would it be possible to invite representatives of the union to join this conversation? I'd really value their input on how we should cover cost-effectiveness, and on the bigger question of their very direct involvement in this project. --Anthonyhcole (talk) 11:43, 7 October 2012 (UTC)

Consumer Reports is based in the United States where the price of healthcare is often a factor in determining what kind of treatment a person can have. This is because the US government does not provide general healthcare in the same way that similar countries do. A physician's use of cost as a factor in recommending healthcare would also apply to countries in the developing world. This practice has two effects; one, poor people often want medical treatment but simply cannot have it, and have to chose alternatives to the first line treatment. The other effect is perhaps more interesting and more discussed at Consumer Reports - it means that in all cases there is greater choice in medical practice, so patients even without medical knowledge have more agency of choice in deciding what their treatment will be and doctors also are more prone to chose treatment which is contrary to best practices set by the government or medical specialty professional organizations. Lots of forces influence such choices, but one example is that in America there is an advertising industry for drugs and treatment and the nature of advertising is to introduce illogical options.

An implication of this for Wikipedia is that people in the United States who are visiting Wikipedia articles are not infrequently using this encyclopedia as a source of information to parallel the information they get from their doctor, the advertising industry, and other consumer literature. Ultimately, some patients in the United States feel some pressure to review their healthcare and participate in decision making. Because Consumer Reports has felt consumer demand for a long time for some health cost information, they have published and managed awareness campaigns to encourage people to talk with their physicians about following the best practice guidelines, and as MistyMorn said in the discussion on the other page which began this one, this is based on a cost-effective analysis of possible treatments.

The way that Consumer Reports does health education is in publicizing problems wherein statisticians determine that many patients are, for some reason, receiving treatment which is against best practice guidelines. Because of market pressures, this seems to happen in favor of consumers somehow choosing to have more expensive, less recommended treatment than the first-line treatment.

I do not understand what the original concern is about the heart device, but in general, I feel that information like cost-effectiveness, when such information exists in reports citing secondary sources which meet MEDRS, has a place in Wikipedia articles because I feel that people are coming to Wikipedia seeking such information. To what extent does that satisfy the original issue? Could the original question be restated? Blue Rasberry (talk) 13:03, 7 October 2012 (UTC)

Briefly: I chose the example of the implantable cardioverter-defibrillator because of the obvious relevance of cost-effectiveness studies, which influence (along with cruder financial considerations) the implementation of this relatively costly option whose long-term clinical effectiveness is supported in the literature. My intention was to explore how this fits in with MEDRS and our broader editorial approach. In the absence of other input, I intended to make a few bold updates to test the water, but I haven't got there yet. However, we could just as well discuss any other clinical intervention where cost-effectiveness analysis is relevant.

(Adding: I've also made a few more general considerations of my own on Anthony's talk page.) —MistyMorn (talk) 13:24, 7 October 2012 (UTC)

Repeating some of what I said on my talk page: Any coverage here of the cost effectiveness of investigations and therapies should be supported by only the best sources, obviously. If highly regarded peer-reviewed journals, or professional and academic societies, or other rigorous and independent bodies haven't published cost effectiveness analysis of an investigation or intervention, then we have nothing to say on the matter until they do. If such sources disagree, we report that. --Anthonyhcole (talk) 14:49, 7 October 2012 (UTC)

I think you're taking it a little too far. If only normal-quality sources address something, then we should report what they say. We prefer high-quality sources over normal-quality sources, but we do not prohibit the use of normal-quality sources when they are the only sources that exist. WhatamIdoing (talk) 17:05, 7 October 2012 (UTC)

How are you defining "normal quality"? --Anthonyhcole (talk) 17:23, 7 October 2012 (UTC)

The comments on the WP:WEIGHT page were helpful for me to understand this. The way I understand it, in a general (non-medical) context, the "best sources" are commonly accepted reference texts, whereas "normal sources" should be prominent and easy to name. All other points do not belong in Wikipedia regardless of whether it is true or not. This is just my understanding, and clearly, an additional layer would have to be added to translate this into the current context. Cheers! Arided (talk) 19:33, 7 October 2012 (UTC)

I guess a relevant case in point of a source which doesn't meet MEDRS (imo at least) but which could potentially fill a gap is this review [22] under discussion below. The journal [23] is not found on PubMed, but appears to be peer reviewed [24] and supported by recognized bodies. Nevertheless, judging from the "excerpts" of the review I've been able to see (here [25]), the quality appears to be rather low. —MistyMorn (talk) 18:34, 7 October 2012 (UTC)

I'd define a "normal quality source" as something that's generally seen as reliable, but not as the very best possible source. In an effort to have a complete article, I'd accept a source that is "good enough" to support the claim rather than "only the best". For example, I'd accept a regular old magazine article that analyzes the cost of getting a child to the dentist under this program vs the cost of getting a child to the dentist under that program. Normal-quality reliable sources include lay-oriented (non-fiction) books, news media, patient-oriented websites run by charities, and so forth.

I'm not actually talking about WEIGHT issues: there really aren't "significant minority viewpoints" about how much it costs to send a child to the dentist, or which antibiotic is most frequently prescribed, or how much revenue BigPharm brought in last year from a given drug. These are pretty simple facts, not "viewpoints" that have "adherents". WhatamIdoing (talk) 20:26, 7 October 2012 (UTC)

Ok, but cost effectiveness analysis is methodologically much more complex (and therefore open to criticism) than simple costing. For example, there's the small question of what clinical effectiveness data to use and how. Observational data (which reflect a particular real-world context) or data from randomized trials (which are strictly generated, but come from an experimental setting which may not completely reflect real-world practice). So I would argue that MEDRS really is helpful here (though I also feel even more strongly that it really should apply to non-medical bio- pages like Biology and political orientation). Also, on a separate matter, I'd never be altogether inclined to take Big Pharma at its word... —MistyMorn (talk) 20:43, 7 October 2012 (UTC)

I disagree with WhatamIdoing that issues such as cost to send children to access healthcare are mere facts and not viewpoints with adherents. Such perspectives could be described as public health, global health, home economics, economics, governance, or some other practical concern. I am sharing information of this sort, for example this recommendation set published by professional organizations promoting a viewpoint about practical matters related to cost and patient safety. They are backing this viewpoint with information from review articles which you can see on the second page. This statement was not published in a peer-reviewed journal, but the statements of top-level professional organizations in their own fields - especially when they cite MEDRS sources, collaborate with external organizations for editorial review, and have a non-profit motive - must be worth something. Blue Rasberry (talk) 15:19, 9 October 2012 (UTC)

question: MEDRS versus NPOV

Hello: I'm having a discussion on Talk:Transcendental_Meditation_research about "contraindications" for that practice. I've found a list of papers which purport to show negative effects here: http://www.behind-the-tm-facade.org/transcendental_meditation-harmful-abstracts.htm However, other editors of that page suggest that they are following WP:MEDRS and that none of the studies I've mentioned can be used (I guess they are not up to par WP:MEDRS standards?). However, I've suggested that since some not-insignificant number of studies exist which purport to show negative effects, that there should be some room to mention this in the article, even if these studies have been disconfirmed by other research, or left out of large-scale meta-studies, etc. Could someone please comment, either here or on that page, about the validity of using WP:MEDRS as a rationale for leaving out individual studies, while citing one "alternative medicine" book instead? Thanks. Arided (talk) 21:55, 6 October 2012 (UTC)

I can confirm that none of the papers listed on the web page satisfies MEDRS. In particular, none of those labeled 'review' appears to have been published in a reputable peer-review journal. Yes, it is important that any biomedical claim should be backed by reliable medical secondary sources, per WP:MEDRS; when such sources are lacking the claim should not be made. This is most certainly not a question of "MEDRS versus NPOV". —MistyMorn (talk) 23:21, 6 October 2012 (UTC)

Actually the first one listed on that page is published in International Journal for Psychotherapy, "a peer-reviewed, scientific journal; it is published three times a year in March, July and November, by the European Association for Psychotherapy" [26] Arided (talk) 08:31, 7 October 2012 (UTC)

But anyway just to be clear: I'm not suggesting that the papers do satisfy MEDRS, my point is that these papers suggest an alternate perspective, noticably different from the one expressed on the current Transcendental Meditation research web page, and that this viewpoint should somehow be included, even if it is not put forth as a "biomedical claim" but rather a sociological claim, i.e. "some researchers have attempted to demonstrate negative effects." That's just a point of fact, as for whether they have been successful in that effort or not I don't know. Certainly there must be a way to flesh out the claims already made on the page (which references Mosby's Complementary & Alternative Medicine: A Research-Based Approach. published by Elsevier), i.e. we could find the papers that are cited by that book, and include them? If there are contraindications, then that should be explained clearly and in sufficient detail, this is my only point! Arided (talk) 08:42, 7 October 2012 (UTC)

Since no paper in the International Journal for Psychotherapy is listed on PubMed (the journal is not indexed by the NCBI [27]), I don't think it should be used to support biomedical claims, per MEDRS. That's as far as I'm personally prepared to go in addressing this question. Imo, nuanced, informative wording such as "Some researchers have claimed..." does not in itself constitute a biomedical claim, and should be subject to WP:WEIGHT and WP:RS. But that would be another question, not directly involving MEDRS. —MistyMorn (talk) 10:32, 7 October 2012 (UTC)

Here's a paper that is listed in PubMed. It's cited by 3 other PubMed articles (and has 74 citations on Google Scholar). One of these papers (Harv Rev Psychiatry. 2009; 17(4): 254–267) says that "Alongside its alleged benefits, meditation may have certain adverse effects. It has been reported that meditation can cause depersonalization and derealization, and several reports have found associations between meditation and psychotic states." It seems to me that something like this could go into the page? Arided (talk) 11:08, 7 October 2012 (UTC)

This paper by Lazarus is already in the article. That's the paper that Freeman cites. Freeman is a proper secondary source, being on the Brandon/Hill list of core books for a medical library (which this guideline says make good sources for medical articles). Lazarus's paper isn't a scientific study -- it's a page and a half, and is completely anecdotal. Freeman puts it in the proper perspective, in a manner similar to other research reviews, such as the Cochrane reviews that also mention it. The Harvard review by Dakwar is already used in the article. If it says something specifically about TM, that can be used. TimidGuy (talk) 11:22, 7 October 2012 (UTC)

OK, I introduced an actual quote and reference to the Lazarus paper, noting that it is anecdotal. I believe these are constructive edits that serve to flesh out the ideas in that section, and they collectively make the point that (1) different types of meditation can have different results, and different people can experience different results with the same types of practices; (2) some people have had adverse reactions to TM in particular; (3) this may have been precipitated by "overmeditating". I think those points are important to communicate in a "contraindications" section. The points seem backed up by the literature, some of which is necessarily anecdotal (because it deals with individual people). If there's a better way to make these points, I'd look forward to including it. But I think that for the moment the claims are sufficiently well sourced, and are relevant enough (per WP:WEIGHT) to include in this article. Arided (talk) 12:37, 7 October 2012 (UTC)

Thanks for bringing this discussion here - these kinds of viewpoints are really helpful and you seem to be engaging other editors in an ideal way. Blue Rasberry (talk) 15:21, 9 October 2012 (UTC)

Primate Autism-Vaccine Study

Could someone with more experience in the field have a look at this article and see if it complies? --Daniel 17:18, 9 October 2012 (UTC)

This seems to be settled, as it now redirects to vaccine controversies. Biosthmors (talk) 18:14, 9 October 2012 (UTC)

Alternative medicine article discussion to restore MEDRS and NPOV content and sources such as Annals of New York Academy of Sciences and Journal of Academic Medicine

A discussion to restore the first 14 sources of this version, including Journal of the Association of Medical Colleges, Annals of New York Academy of Sciences, Academic Medicine, Canadian Medical Association Journal, Medical Journal of Australia, Nature Medicine, etc., to the Alternative medicine article is now going on here. ParkSehJik (talk) 02:57, 22 November 2012 (UTC)

Discussion re uniform application of MEDRS standards to all WP articles

FiachraByrne wrote in supoport of removal of dubious and citation needed tags -

"Psychiatry is one of the oldest medical specialisms. It's designation as medical practice is a disciplinary/professional attribute that has little to do with the actual content of psychiatric knowledge or the nature of psychiatric practice. To establish this it is unnecessary to evaluate whether in any or all instances psychiatry adheres to the so-called 'scientific method'."

At Wikipedia, having a scientific basis is a WP:MEDRS issue, not just a matter of determining the common usage on the street.

There is rigorous enforcement of WP:MEDRS by hawks (of which I am one), re assertions re TCM being healing "medicine", as defined in that article and by MEDRS standards. The only allowable edits are that TCM practitioners "claim" to heal. MEDRS should be interpreted to require similar qualifications to psychiatry related articles, and all medical claims at WP, especially if MEDRS or RS can be provided to base it on.

The same WP:MEDRS standards should be applied to psychiatry as to alternative medicine articles. Traditional Chinese Medicine (TCM) is also one of the oldest "medical" practices.

My dubious tags and citation needed tags were removed citing “common sense” trumping my RS and MEDRS based content that questions Diagnostic and Statistical Manual of Mental Disorders, re the designation of psychiatry always being medicine, and not just some parts of it, with the associated implications of established efficacy in healing real diseases, and calling for citations for unsourced medical claims.[]

After my citation needed tags were removed, I deleted the medical claims altogether citing MEDRS, as I would at any alternative medicine article. The content was restored without sourcing[28] as being "pretty basic".

My construction tag was removed[29].

My MEDRS and RS edits under WP:MOS (lede) “include significant controversy” were deleted as violating WP:BATTLE and “WP:POV to absurdity” in the forensic psychiatry arricle -[30]. Similar deletions of content occurred in the psychiatry article.

Allen Frances, chair of the DSM-IV Task Force - "DSM 5 will accept diagnoses that achieve reliabilities as unbelievably low as 0.2-0.4 (barely beating the level of chance agreement two monkeys could achieve throwing darts at a diagnostic board."[31]

“Common knowledge” should not trump MEDRS to rewuire sourcing for medical claims in psychiatry related articles, especially if MEDRS exists challenging that "common knowledge. MEDRS should be applied with the same rigor in psyhiatry related articles as in alternative medicine articles.

The psychiatry, forensic psychiatry, Bipolar disorder, be qualified under MEDRS to say that part of psychiatry is medicine, and part not based on science, and part not intended to heal.

Diagnostic and Statistical Manual of Mental Disorders V should not be the only MEDRS source allowe in, and MSM V might not meet MEDRS standards as per the quote above.

Please comment and discuss ParkSehJik (talk) 20:57, 29 November 2012 (UTC)

It is unhelpful to begin a discussion thread with a loaded question then provide a wall of text other editors are unlikely to read. Your postings fail to follow POV. Certainly there are problems with psychiatry, but your edits present them with undue weight. For example you argued that psychiatry was a pseudoscience by misrepresenting sources at Talk:Pseudoscience#Is psychiatry a pseudoscience? TFD (talk) 20:16, 29 November 2012 (UTC)

Note: I shortened and modified my opening comment per Four Deuces comment. ParkSehJik (talk) 20:57, 29 November 2012 (UTC)

@Four Deuces, the paradigmatic example of pseudoscience by Karl Popper in Conjectures and Refutations was Freudian psychiatry (which is still sanctions as psychiatry for basing prescriptions and expert testimony on) as being unrefutable. ParkSehJik (talk) 20:57, 29 November 2012 (UTC)

He's not exactly beginning the discussion, it was moved here from Talk:Psychiatry per discussion there.

No one should really care where the discussion is held, so long as it is outside of articlespace and notice is provided in the likely places of interest. That said, I would suggest that wikipedia talk:WikiProject Medicine/Psychiatry task force would have been the most appropriate place. Now that it is started, though, just leave it be here, as there's nothing really to gain by shifting fora.

On the original question, no, the standard should be just as high for all specialties: we use the best available secondary sources reflecting current science. Some of the detailled guidance at wp:MEDRS will, though, need to vary slightly. For example, the indexing of psychiatry and psychology papers in PubMed is rather spotty in comparison to other specialties, while other indices may be found to be more comprehensive. Accordingly, lack of indexing in PubMed should not be taken as sufficient reason to dismiss a psych journal. LeadSongDog come howl! 20:51, 29 November 2012 (UTC)

So what is the suggestion? WP:MEDRS applies to psychiatry as it does to the rest of medicine. Doc James (talk · contribs · email) (if I write on your page reply on mine) 21:05, 29 November 2012 (UTC)

(edit conflict)We need to establish equivalent guidance on indexing, based on the best practice in the field. What indexes other than PubMed are comparably authoritative? PsychInfo seems more relevant to psychology than psychiatry, but that at least might be preferable to broader indices such as EBSCO or Scopus: those don't seem to be very selective about the journals they index. LeadSongDog come howl! 21:55, 29 November 2012 (UTC)

If I may I'd like to clarify the context for the quote used above. I had made the point in response to the placing of a dubious tag after the statement in the Psychiatry article that "Psychiatry is the medical^{[dubious – discuss]} specialism ..." My point was that whether or not psychiatry or any other specialism falls within the province of medicine is not due its scientific content but is the outcome of historical processes relating to the institutional and professional recognition of the discipline amongst other factors. It is not a statement that psychiatry does not have a scientific content just that that is irrelevant to its status as a medical specialism. FiachraByrne (talk) 21:32, 29 November 2012 (UTC)

ParkSehJik is again misrepresenting sources by saying, "the paradigmatic example of pseudoscience by Karl Popper in Conjectures and Refutations was psychiatry...." Here is a link to the article, pp. 44ff. Popper clearly mentions psychoanalysis and Jung's individual psychology and not psychiatry in general, which I have clearly explained to him. Claims of modern psychiatrists, such as the effects of anti-psychotic drugs, are clearly "refutable", i.e., it can be shown whether or not they work. TFD (talk) 21:42, 29 November 2012 (UTC)

(Following Four Deuces correction, I changed my comment above to "Karl Popper in Conjectures and Refutations was Freudian psychiatry (which is still sanctions as psychiatry for basing prescriptions and expert testimony on) as being unrefutable." )

Some claims are clearly refutable, some are not. How do you refute a bipolar diagnosis, which is DSM a lifelong condition, if there is at least one episode at one time. The only way to ever refute such a diagnosis is to diagnose the one episode as another disease. But once diagnosed, there is no refuting that there is a disease. That is the fundamental MEDRS problem, psychiatry is a strange hybrid. Some of the the content of the RS sources deleted by others was specific statement that some of psychiatry is based on science in any way. That is the reason for needing qualifications per MEDRS. It is the scientifically legit parts of psychiatry that are worn by epert forensic pschiatrists to cloak themselves as basing their opinions on science, when they are not. Per MEDRS, this needs to be clearly stated. ParkSehJik (talk) 22:47, 29 November 2012 (UTC)

As far as I am aware, reliable sources treat psychiatry (generally) as a medical science and I would expect WP:MEDRS standards to apply to sourcing for relevant content. As has been pointed out, psychiatry is not psychology. A psychiatrist must obtain an M.D. (at least in the USA), can prescribe medication, and the FDA regulates the medication prescribed. To be sure, there are critics who scoff at psychiatry, just as there are critics who scoff at "Western medicine," but a relative handful of critics of a subject do not outweigh the overwhelming majority of reliable sources. Per WP:DUE under the Wikipedia policy of WP:NPOV, we write articles based on the broad consensus found across most reliable sources, and we develop content and sourcing guidelines to support the development of articles along policy guidelines. Zad68 22:14, 29 November 2012 (UTC)

Zad, the reason the issue arose is that in the alternative medicine article, editors with a martketing agenda have been trying to apply the cloak of being "medicine" to their fields, and medicine being based on scientific method gained evidence, thereby as having the appearance of being science based, the very essence of pseudoscience for fraud. See the discussion trying to water down the lede first paragraph at alternative medicine. The same can be said of forensic psychiatry, in which experts can be found for almost any position re themental state of a person in question. This has nothing at all to do with medicine as healing, and wears the cloak of science based respectability, which with a 50-50 split on experts in each court case, is the opposite of statistical significance to base a scientific conclusion on. Furthermore a big chunk of criticism of psychiatry is the opposite of critcism of western medicine" (i.e., evidence based medicine). Karl Popper coinded the term "pseudoscience" and used psychiatry as the paradigmatic example of that which does not lend itself to refutation. Note that DSM gives no method to UN-diagnose anyone, e.g., re bipolar, and declares with test and systematic review sources that are not in existence, a "lifelong" nature of disorders like bipolar, interesting in what the metholological basis might be, since bipolar has not been around as a diagnosis for an average lifetime. I would suggest MEDRS demands, given this, a qualification such as

"Psychiatry includes the practice of medicine specifically to heal mental diseases and disorders, associated medical sciences (not to go in article, but e.g., use at Gauntanamo), applications of theories of the mind in an effort to heal hypothesized mental disorders, and formation of partisan opinion in legal cases where a person's mental state is in question."

This is not a final proposal, but it improves things toward MEDRS standards. I put it up, so others can shoot it down, but it is a start to further improvement. ParkSehJik (talk) 22:40, 29 November 2012 (UTC)

I accept that the US has poor regulation of the professions, US courts have low standards for the admission of professional witnesses and psychiatry has been abused. However these criticisms apply to all sciences and has nothing to do with whether or not psychiatry is scientific. If you want to build a factory that will pollute the environment and kill the people around it, you can find a professional witness to say it is safe. That does not mean that environmental sciences are pseudo-sciences. Doctors told us smoking was good for us, that does not mean that medicine is a pseudoscience. TFD (talk) 23:05, 29 November 2012 (UTC)

It doesn't even matter whether psychiatry is "scientific". The dispute is about whether psychiatry is "medical". Psychiatry can be fully "medical" without being "scientific". Medicine never has been purely scientific, and it never will be. WhatamIdoing (talk) 03:19, 30 November 2012 (UTC)

Park, please consider The Four Deuces' response carefully, it's well-said and dead on. Also I'd like to add by giving you an analogy regarding your proposed definition: Should 'airplane' be defined as 'A propelled, winged vehicle for carrying passengers, equipment, and for flying into buildings in the commission of terrorist attacks'? Although it's true that some airplanes have been misused that way, it's obviously ridiculous to include the abuse as part of a definition of it that includes its intended use. Psychiatry isn't designed or intended to be abused the way it has been, and such abuse should not be so closely tied into the definition Wikipedia presents for it. Zad68 23:22, 29 November 2012 (UTC)

See here I feel forced to include 170 Cochrane Reviews of treatments for mental illness, simply because ParkSehJik decided Cochrane was great when he liked what he thought was their definition of CAM. Turns out that was wrong and Cochrane defines CAM with the Potter Stuart method (I know it when I see it). But meanwhile they looked at many drug treatments for mental illness as well as dance therapy, Ayurveda, and so on. Psychiatry is not entirely a science but then neither is medicine. The distinction is most important in deciding what the taxes pay for and what the legal system allows. However, psych has come under far more scrutiny as a science than CAM even in treating mental disorders. Read some reviews! S B H arris 23:41, 29 November 2012 (UTC)

@Sbharris - You are correct that I have recently completely changed my opinion of Cochrane. When I first saw the Cochrane definiton of alt med as essetially being "scientific truth is that which is pushed on others by the fashion of the time and place sensitive politically dominant". I was somewhat taken aback when I saw that. There is an edit on an evidence-based-user's talk page in which I compared it with Stalin's declaration that the truth genetics is a matter of political dcision, not scientific testing. I pointed to the irony of it being similar to what is described as an essential point in the Annals of the New York Academy of Science re cultural relativism, and I compared it to Stalin's declaration re gentetics whereby its truth was based on political declaration rather than scientific test. (I also pointed out the irony of debating inclusion of the NYAS content, using WP:consensus, when the article was about how preposterous determining truth by consensus is. (@Sbharris - I am beginning to look forward to the subtle humor in your edits insofar as they make me feel ridiculous in my own inconsistencies. I now wonder, did Potter stuart ever comment on the exitence of penis envy, and as to whether Freudian posychiatry might be outlawed as obsence on its face? If he knows it when he sees it, what did he WP:Spade say about all forensic psychiatry?)

More @SBHarris - You suggested I read the studies. I started. Its only one trial , but I hope you get the same chickle out of my suggestion for further research - "Ayurvedic treatment ... versus antipsychotic... Insufficient evidence to support or refute the use of Ayurvedic treatment (combination of different herbs), when compared to antipsychotic medication... "[32] - So antipsychotics are so far as effective as ayurvedic so far, indicating the need for further studies on antispychotics . LOL.

Answer: To be fair, they left out of summary the finding that there was a statistically-significant smaller chance of no-improvement on the Ayurvedic treatment than the chlorpromazine. see here Possibly that's just a multiple comparisons artifact, but I couldn't find any place where Ayurveda came out ahead, except that it produced less sleepiness (at the cost of more nausea). These are small studies. Meta analysis of larger ones come out in favor of chlorpromazine over placebo. It's right close by the one you quoted: here. Perhaps larger trials of Ayurveda are needed. But without a single hint from a smaller trial (as we see for chlorpromazine, even in the Ayurveda trial) it's going to be hard to find funding for them. S B H arris 01:51, 30 November 2012 (UTC)

No trials of Ayurveda are needed. I wish my tax and insurance money would stop indirectly going to fund this nonsense. Re psychiatry, A. I have no doubt at all that much of it is real categoris of disease, science based, sincerely based on an effort to help people, etc. B. I further have no doubt that for any reasonable classification scheme for mental states based on measurable thresholds, there can be found evidence based psychtropics to lessen the symptoms whereby the threshold is not crossed. The question is the areas outside of A, which if they exist, will automatically, be those any attorney whose client is wrongas a professional duty to the client, whereby a forensic psychiatrist specializing in the pseudoscience aspects can be found. Furthermore, if by mere description of outside the norm gets one diagnosed as diseased, I have not doubt the forensic pyschiatrist can be found to diagnose, say, Sbharris disorder, which you would certainly be mentally ill with, and ParkSehJik disorder, which I have, but do not suffer in any way from. This pseudoscience aspect is sufficiently still prevalent, albeit possibly not as bad or bad in a different way, as during the peak of Freudianism. This high pseudoscience content is referred to in many RS and MEDRS. I tried to put it in the article backwards, by citing problems with forensic psychiatry sources, when that is puttin the cart before the horse as i comment below. Now I need to go find new sources, and I believe if sufficiently MEDRS, will have no problem going in, and which will lead to correlary sources and content for the daughter articles of psychiatry. ParkSehJik (talk) 02:04, 30 November 2012 (UTC)

More seriously, I cannot (so far) find a single study cited that empirically ties "disease" to the mental state, only that the symptoms associated with that state were often significantly diminished by the intervention. As an example of my point, it it is considered a mental disease to speak out against serious wrongs by civil disobedience, then administering thorazine would significantly reduce that, by flattening the mind of the protester. That does not indicte it really was a disease, just that there is an intervention to diminish certain symptoms. And I am still searching for studis associated with involuntary commitments, etc. ParkSehJik (talk) 00:49, 30 November 2012 (UTC)

Yes, TFD did hit the nail on the head re basing a criticism section of psychiatry on RS for criticisms of forensic psychiatry. That cannot be done because it is putting the cart before the horse. The reason forensic pschiatry stands out as qualitatively worse than other expert witness (WP:SPADE explative deleted) testimonu is at the end of the specific proposal I made here - Proposed content for criticism section. A related call for RS re current allegations of pseudoscience is here RS for WP:SPADE "a pseudoscientific trashcan diagnosis, to provide an FDA-approved “indication” for the prescription and marketing". ParkSehJik (talk) 01:34, 30 November 2012 (UTC)

Whether something is properly called a "disease" depends on your point of view. There is no authoritative, universally applicable definition. There are culture-bound syndromes in which a healthy person calls himself diseased, and psychotic and neurologic syndromes in which a person who appears diseased to others calls himself perfectly sick. For example, many people with bipolar disorder believe themselves to be perfectly fine (it's the rest of the world that is screwed up). Whether the rest of the world should pay any attention to the unsupported and unprovable assertions of mental health from people whom the world has decided are unable to see their sickness is a conundrum. WhatamIdoing (talk) 03:19, 30 November 2012 (UTC)

Agreed, when you get down to the ontological status of physical diseases - although largely irrelevant to medical practice - things can become pretty confused. Howsoever objective and impartial observation and measurement may be, the designation of "something" as pathological is ultimately a value judgement. Psychiatric objects suffer greater instability than most as, oftentimes, it is harder to construct objective criteria to police the boundaries between normal/abnormal, healthy/pathological, etc. That aetiology is poorly understood opens further space to dispute the nature and applicability of psychiatric diagnoses. FiachraByrne (talk) 13:22, 30 November 2012 (UTC)

There is either not a proposal here or multiple proposals here. In any case, there are many people who are on hand to discuss MEDRS if anyone ever wants to do so. It is not a controversial subject and I know of no one who has put forth an argument that it is not a sensible guideline for content in the field of medicine. Blue Rasberry (talk) 19:03, 30 November 2012 (UTC)

Social constructivism, objective reality, science, medicine, disease

Social constructivism and knowledge in science

Let us begin with some stipulations. Language and ideas define how we think. All scientific concepts are social constructs, from the idea of quarks and electrons to classificational things like the idea that humans are members of the family Hominidae along with other apes. Some of these ideas cause outrage, but it will do no good to complain that there’s no way to get humans OUT of Hominidae once they’ve been classified there. Actually there is, but it will take consensus of taxonomists and this classification has lasted 187 years. It will do no good to complain that reality isn’t determined by consensus. In fact, it is. Reality here, is what language terms are useful to us. If I identify an “electron” it tells me things about what will happen when I apply a voltage to it, and so on. There’s no point in wondering about its ultimate “reality.”

Disease as social construct

In medicine, “dis-eases” (see etymology) are also social constructs. That’s not only true of mental diseases. Is dwarfism a disease? Who are you to define the pathologic limits of height? Is progeria a disease? Who are you to define what the correct aging rate should be? Homosexuality? Wups. Dis-ease implies discomfort or pain, but who is to say that discomfort and pain aren’t natural parts of the human condition, like cancer? There’s not a disease you can name that you can prove SHOULD be universally regarded as a disease, without a lot of prior assumptions that are all open to question. Death is natural. Something causes dead tissue? Who are you to say that all tissues are (or even should be) immortal, and that we should regard necrosis as a pathology? It’s just prejudice. Or a self-fulfilling definition of “pathology” which won’t hold up so scientific scrutiny. You say it smells bad, but that’s your personal opinion. I say it’s basically an esthetics question.

In medicine, we try to make our social construct “diseases” serve medical purposes. They should be predictive, or they aren’t really useful (non-predictive diseases don’t reduce medicine even a bit to a “science” or praxis). So a disease needs to carry a prognosis and (hopefully but not necessarily), imply a treatment. Depression (whatever you call it—it has a different name in Chinese) implies a higher risk of suicide. So it’s a useful term. Not all mental illness have definitions that cross all cultures, but then few ideas do. So what? If we find a culture that only counts “1, 2, 3, many”-- that’s no reason for us to give up our idea of “100.” It’s too useful.

Mental illness as disease

As for fixed delusions, we take those into consideration when they involve our safety. Joe Schizo may think that paperboys are agents of Satan who need to blood-sacrificed if they trespass onto his lawn. Are we going to philosophize with him, and wonder if he has a mental “disease” or not? No. He’s a danger, and we’re going to use force on him since he’s outvoted. It hardly matters what we call his mental state, except by prior experience with others like him. If he suddenly believes these things because he hears voices that tell him that (rather than sees visions of it), that puts him in a category we have lots of experience with, and we may not need to imprison him. There may be drugs that help, that can avoid that. So again, these categories are useful. Useful even to Joe (as we presume he’d like to avoid institutionalization).

The courts and abuse of knowledge

A word about the courts and natural science. The courts are full of lawyers and judges (and judges are mostly just lawyers in black robes). And yes, people too dumb to avoid jury duty, or with too little to do. Unless the people in front of the bar are patent attorneys, they probably know little of the natural sciences. And those scientists who have lasted into jury duty have probably been excused for knowing too much. Hence, courtrooms operate much like Wikipedia. These people look for reliable witnesses and then listen to them. But they have no way to judge the reliability of witnesses except what other witnesses say about them. It’s all credential and hearsay and precious little science. For example, we’ve long known that eyewitness ID of strangers is worthless, but courts still place great confidence in it. Courts have no “meta-strategy” of abandoning bad previous habits of epistemology. Nor does Wikipedia. We want reliable sources but our only approved way to identify them is that other reliable sources SAY they are reliable. But that game has no end, and it runs on prior habit. It has no ultimate appeal to experiment, as happens (ideally) in the natural sciences (save for string theory, heh). So Wikipedia produces a product much like the legal system, because it’s set up that way. Does Jimbo Wales look like a scientist to you? No? Then stop whining.

Agreed, psychiatry is abused by the courts-- but then every profession is abused by the courts. Because courts don’t run experiments. They just listen to “experts.” Picked out by other experts. For that matter, ALL of forensics (from scientifically well-agreed-on things like fingerprinting, on down to the worst things like line-ups) has a deplorable amount of scientific good-experiment backing. This is not the place to complain about that, either.

Wikipedia and abuse of knowledge

Unless you can find experts to testify on your side, at which point the WP “court” may allow your witnesses.

But don’t expect much from the jury. The courts are kinder to small-minority views than is Wikipedia, since even WP:SNOBALL guilty or crazy people are given nearly equal time by the courts. But not on WP. If you think NPOV guarantees that, you haven’t read it carefully.

Conflict of interest

Courts and WP try to sniff out conflict of interest (see WP:COI), but they aren’t good at it. Cochrane has little conflict of interest in medicine, as nobody pays them. But NCCAM, the National Center for Complimentary and Alternative Medicine at NIH will get $127 million dollars of your tax money next year, and that’s a rather big incentive for them to believe they have something important to find with it. You’re going to tell me that M.D.s have a huge reason to believe in “standard medicine” (over AM) but they really don’t. If other stuff worked, they could very easily adopt it as CAM (believe me, it’s a lot easier regulatory-wise for me to set up to stick sterile needles in you randomly, than it is for Joe Alternative or Joe TCM and I defy you to find me studies that show it won’t work just as well).

The big money interest in AM is in the AM practitioners, who cannot do CAM and have to practice AM, by and large, or they have nothing. Licensed physicians, subject to their medical boards, are given much broader leeway to do CAM than people who want to give you herbs or IV chelation without benefit of a sheepskin. S B H arris 03:37, 30 November 2012 (UTC)

My biggest problem with NCCAM is too many consonants in a row to pronounce in one syllable, which is already more than it should be alloted as its share of syllables per share/value. Here at Wiki this is MEDRS COI - COI re Bipolar disorder and Bipolar spectrum at WP? ParkSehJik (talk) 04:18, 30 November 2012 (UTC)

They definitely could have come up with a sexy accronym like all the med studies these days. Even NASA missions. MESSENGER is a clever reference to Mercury. S B H arris 06:19, 30 November 2012 (UTC)

WTF

Has this page turned into a blog or something? Move your mouse and press down on the scroll bar thingy. Drag till you get to the top of the page. What is the bold text in the orange box?

This is the talk page for discussing improvements to the Identifying reliable sources (medicine) page.

Can any of the authors of the above two sections summarise in a couple of short sentences what changes/improvements they wish to make to the guidelines page?

Thought not.

Please take your addled thoughts and tiny verbosity elsewhere.

Colin°^Talk 13:50, 30 November 2012 (UTC)

The section above here has a discussion between 8 people which seems to lack any constructive and concise input from you. At the center of it is a problem of epistemology. What do we mean by "reliable"? MEDRS, which is supposed to be about locating reliability, does not define it. Okay, I propose concretely that you all think about it and add it to MEDRS, in order to get a sense of the problems in dealing with psychoanalysis and alternative medicine. It isn't really defined at WP:RS, either. Except by reputation, which misses the whole problem, due to being highly subjective. Which highlights a basic problem at WP. Try to come up with honest and workable answers to these questions and see how far you get. S B H arris 21:11, 30 November 2012 (UTC)

When we need a definition for reliable, we can simply go to WP:RS: "Reliable sources may be published materials with a reliable publication process, authors who are regarded as authoritative in relation to the subject, or both. These qualifications should be demonstrable to other people." --Ronz (talk) 22:32, 30 November 2012 (UTC)

I have no idea what a reliable publication process is. Some homeopathic journals are peer reviewed-- by practioners of homeopathy. And they have ways of demonstrating their knowledge of homeopathy to each other. Same as I might demonstrate a comprehensive knowledge of the Star Trek universe or Mormonism. None of which has anything in the least to do with reality. Unless you believe these things can be studied academically, like literature or theology. The natural sciences, however, require more. MEDRS should have the extra but doesn't really articulate it in a way as to keep homeopathy and crystal healing and voodoo out. Worse still, it claims to apply to alternative medicine! We need a NATSCI-RS for WP. What you read in RS was written to cover news stories in the Washington Post, not evaluate medical claims. Yes, MEDRS gives good ground rules for evaluating the truth of medical claims, but nothing about why WP editors should abide by them. Which is why CAM pushers cry foul. Nobody is interested in "truth" in other WP areas. So why so hard-nosed here, all of a sudden? It's a fair question. Without some basic demand that testable claims must have been fairly tested, WP has no more right to judge medicine against alternative medicine than it does Scientology vs Roman Catholicism. S B H arris 01:18, 1 December 2012 (UTC)

We have WP:SCIRS. It may only be at essay-status, but still extremely useful. --Ronz (talk) 05:30, 1 December 2012 (UTC)

Ah, I hadn't seen that. It's a start though I disagree with a few particulars (college undergrad texts are often not careful and disagree on basics that are only cleared up in upper division or grad texts). And since this essay still relies on experts, it doesn't tell you how to ignore an expert in homeopathy vs one in pharmacology. Many reviews will simply ignore pseudoscience with untested or (worse ) untestable claims. But not all. The crazies have their own reviews . There are textbooks of homeopathy. All of it would fail a James Randi challenge but even scientists can be taken in by True Believers. WP has no mechanism to tell plausible CAM that doesn't require extraordinary evidence since it doesn't have extraordinary claims (e.g. herbalism) from the nutcases. Medicine should have a Carl Sagan, and instead we got an Andrew Weil. Possibly due our bad karma. :)) S B H arris 06:05, 1 December 2012 (UTC)

Is DSM MEDRS?

Does DSM meets WP:MEDRS standards to be included as a source?

DSM IV is 20 years old, an eternity in the rapidly advancing fields of medical science. Allen Frances, the Chair of DSM IV wrote a "scathing editorial " (language of a major autism organization) critical of DSM V as to its methodological basis - Opening Pandora's Box in Psychiatric Times. Similar editorials by prominent psychiatrists appeared in major newspapers globally. The coverage and responses were so extensive that it became widely covered even in the popular science press.

This month, Discover Magazine named the story as "Top 100 Science Stories of 2012". Six months ago, Discover published about the controversy - The "Bible of Psychiatry" Faces Damning Criticism—From the Inside From that story -

"...a psychologist and psychiatrist who were members of the DSM-5 work group for for personality disorders, found that the group ignored their warnings about its methods and recommendations. In protest, they resigned, explaining why...

'The proposal displays a truly stunning disregard for evidence. Important aspects of the proposal lack any reasonable evidential support of reliability and validity. For example, there is little evidence to justify which disorders to retain and which to eliminate. Even more concerning is the fact that a major component of proposal is inconsistent with extensive evidence…This creates the untenable situation of the Work Group advancing a taxonomic model that it has acknowledged in a published article to be inconsistent with the evidence."

Is DSM MEDRS, and if so, how do we tell which parts of DSM are MEDRS, and which parts are "inconsistent with extensive evidence", and thereby violate MEDRS? ParkSehJik (talk) 17:25, 10 December 2012 (UTC)

Absolutely, yes, the DSM meets MEDRS. Calling psychiatry alt med is ludicrous. Yobol (talk) 01:08, 10 December 2012 (UTC)

Re "Calling it alt med", more "ludicrous" is the inclusion of Ayurveda by psychiatry History of psychiatry in India. ParkSehJik (talk) 16:49, 10 December 2012 (UTC)

Was it alt med in 1973 when DSM II defined homosexuality as a psychiatric disease? [33]. Yes, it was 302.0 Sexual orientation disturbance (Homosexuality). That means psychiatrists could bill to treat it! But then they decided it wasn't really a mental disease. Now, we think you have a disease if you think it's a disease (302.666 Republican political orientation-- no known drug treatments yet for this, except highly illegal ones).

Presently DSM V is deciding if they want to classify Premenstural Dysphoric Disorder (PMDD) as its own separate category and disease, and not just a subtype of not-otherwise classified depression, as DSM-IV now does. Yes, it's pretty much PMS on steroids (no, not those kind of steroids-- the other kind-- no, the OTHER kind), and there's big debate. And worse, the name "mad cow disease" has been taken already.

Here's a fun fact: Did you know that all this psych stuff is in the Dewey Decimal System 100's? That's a fact from Library science. Which in some ways is more related to the formal sciences like math and computer science and linguistics, but is not really a natural science. The differences are interesting.

Physicist Richard Feynman has a charming story about this father who told him a bird's name in six different languages, all bogus. Until Feynman got the point that you can know the bird's name in any number of languages, and still know nothing about the bird itself. How it will behave, and so on. All you know about, without that, is human culture, not about birds. A point that had been made long before by Ernest Rutherford, by the way.

So this is our problem here: how much of psychiatry (and indeed medicine itself) is about human culture, and not about, well, the expected behavior of that interesting "bird"? Wikipedia is required to reliably cover BOTH human culture AND the knowledge that comes from the natural sciences (i.e., what makes engineering work, whether you believe in it, or not). But WP should be careful not to try to mix them up more than it has to. Sometimes I think these shrinks aren't so careful. What do you think? Sexual orientation disturbance, anyone? S B H arris 01:57, 10 December 2012 (UTC)

Yes DSM is a reliable source of psychiatric information. Doc James (talk · contribs · email) (if I write on your page reply on mine) 03:34, 10 December 2012 (UTC)

What is "psychiatric information"? Information on psychiatric problems, or information on how the majority of psychiatrists think about those problems? Since they aren't the same thing. Again I have to point you to the decision on homosexuality in late 1973, above, which was qualified historically with this: If homosexuality per se does not meet the criteria for a psychiatric disorder, what is it? Descriptively, it is one form of sexual behavior. Our profession need not now agree on its origin, significance, and value for human happiness when we acknowledge that by itself it does not meet the requirements for a psychiatric disorder. Similarly, by no longer listing it as a psychiatric disorder we are not saying that it is "normal" or as valuable as heterosexuality. Oh, goodness, no. We wouldn't go that far, gasp. Of course it isn't "normal" and of course it's not "as valuable." It is less valuable. It is disvalued. This is the official 1973 opinion of the American Psychiatric Association task force for rewriting DSM-II (edition 6).

As I mentioned on the other page, a similar thing happened in 1980 when DSM-III came out and psychoanalyists found to their horror that all the Freudian neuroses had disappeared out of official APA diagnoses, along with presumably the subconscious conflicts causing them. The key problem was that you could not bill a third party payor for treating somebody's subconscious conflicts. You see. It sounded too much like billing for fixing a chi'i imbalance. And thus the True Purpose of DSM in all of its later incarnations is not defining science, but defining what you can bill insurance and Medicaid and Medicare for. These all require a diagnosis with a number, and it had better sound official.

This is not to say that psychoanalysis died in 1980, but it changed its terminology and it went to being a more cash-for-service industry, like any entertainment industry. But for that matter, if you take cash, nobody really minds if you pay somebody to spend time rerouting the prana energy flow to your fifth chakra, either. A little lower, please. Lower...

Anyway, the point is that if a lot of the stuff in the DSMs is not backed up by valid natural science experiment, then why should Wikipedia pay any attention to it, except as it does for authoritative sources of religious beliefs? There are sites you can go to, to find authoritatively what the Roman Catholic Church believes (for example), and they are valid per WP:RS and WP:V. But they wouldn't pass WP:MEDRS, and a lot of psychiatry wouldn't. And in fact (as has been mentioned here) a fair fraction of standard medical practice wouldn't, either. I can remember when we gave everybody with spinal/neurological injury a big dose of IV methylprednisolone. No more. I can remember the era of the Swan Ganz catheter for just about everybody with heart problems and shock (now you can hardly find one-- where did they go?). I can remember plain metal stents for coronary disease (wups-- worse than useless in the long term). Now we have new and improved stents! Use them quickly! And don't you dare call those old plain ones "alternative." Or the Swans, either. The lesson is that medical treatments, like literary criticism, need to stand the "test of time." Sometimes that decision is forced too quickly, even in medicine. For example, whenever you see a "consensus recommendation" in any journal, you know the real reason for it, is that there actually is no consensus! Have you noticed? If there was, we wouldn't need the article. S B H arris 04:21, 10 December 2012 (UTC)

I don't think that this sort of essay is helping resolve the dispute, which seems to center on whether or not psychiatry is accepted by mainstream medical practitioners, and especially whether a Wikipedia editor's personal beliefs about whether it should be accepted as part of mainstream medical practice (e.g., by reading definitions of the terms and evaluating the field's failings) is sufficient to write the article to reflect the editor's personal beliefs, or whether he's going to have to either find a substantial body of literature that calls psychiatry "pseudoscience" or "alternative medicine" or give up on adding his personal beliefs to that article. WhatamIdoing (talk) 07:11, 10 December 2012 (UTC)

The "dispute" is how to tell which parts of DSM are MEDRS, and which parts are inconsistent with extensive evidence. DSM IV is 20 years old, a very long time in the rapidly evolving field of medical science, and has many things in it that are outright contradicted by extensive evidence developed since it came out. DSM V will knowingly contain diagnoses "inconsistent with extensive evidence". How do we tell which parts are MEDRS and which are not? I do not know the answer to my question. ParkSehJik (talk) 17:17, 10 December 2012 (UTC)

You don't seem to understand sourcing on Wikipedia. The DSM is typically used to cite one thing: what the DSM says. That kind of sourcing is appropriate for just about any text in any article. We typically separately cite the WHO for what the ICD says. Then a well written medical article will cite secondary journal reviews for any additional information. The question here makes no sense. Criticism of the DSM is welcome in the DSM article, as is criticism of any reliable source (some people don't like the New York Times or CNN, for example). The DSM is most certainly a reliable source for what the DSM says. Please read WP:V; if you disagree with Wikipedia policies, take it up there. If you disagree with secondary high quality recent published reviews used to cite text about conditions that happened to be mentioned in the DSM, that's another issue. SandyGeorgia (Talk) 18:01, 10 December 2012 (UTC)

@SandyGeorgia - Thanks. As I understnd it, at Wikipedia, there is a difference between this edit

"Observing A being over threshold B indicates disease C" (with DSM as the MEDRS ref)

and this edit -

"According to DSM, observing A being over threshold B indicates disease C" (with DSM or a secondary reference to DSM as the source)

How do we know which kind of edit to make, given the abovestated problems with DSM?

Moreover, given the more detailed analyses of DSM V in the literature, there is a big different between

"Behavior over threshold A is a mental disease." (sourced by DSM), and

"According to DSM, Behavior over threshold A is a mental disease."

Do we simply write that something is a disease, or do we have to qualify it if DSM is the source, per MEDRS? ParkSehJik (talk) 18:26, 10 December 2012 (UTC)

I don't think I ever describe something in any of the terms you've used (as a "disease"), so the example isn't highly relevant to my day-to-day editing. We say things like, the diagnosis is conferred according to the DSM when X, Y and Z are met, and the ICD when A, B and C are met. Your language isn't language I would use in any article, so the way you are framing the issue may be part of the problem. We state what the diagnostic criteria are and according to whom. That is WP:V. The problems you are pointing out with DSM (that it is criticized) apply equally to just about every single reliable source, whether that's the BBC, the New York Times, CNN, whatever. Our standards for reliability are laid out at WP:V and WP:RS-- MEDRS merely shows how to apply WP:V in medicine. The question you are posing here makes no sense and gives the idea that you don't understand the fundamental policy of verifiability. WP:UNDUE seems to be a problem area as well. SandyGeorgia (Talk) 18:41, 10 December 2012 (UTC)

@SandyGeorgia - Built into their names is an implied classification as "disease" (or in the language of psychiatry, "disorder"). For example, bipolar "disorder", or "Mood 'disorder' otherwise unspecified" and "General psychiatric 'disorder'". ParkSehJik (talk) 18:54, 10 December 2012 (UTC)

That's why I suggested your problem might be with WP:UNDUE (we can't ignore the preponderance of reliable sources). You, or I, may not personally think of lots of neurobiological conditions as disorders, we might not think that a cold is a "sickness", but the preponderance of high quality reliable sources do, and our personal opinions when editing Wikipedia are not relevant. Criticism of the way the DSM is written is not the same as the implication that medicine doesn't view these conditions as deviations from normal, or invalidate the rest of the research that goes in to the conditions and is cited in our articles. If you dislike the names of the conditions, your problem, then, is not with MEDRS; you have taken over this page (and others) with what appears to be a matter of fundamentally misunderstanding WP:V and WP:DUE. And it doesn't belong on this page any longer. SandyGeorgia (Talk) 18:59, 10 December 2012 (UTC)

(edit conflict)

We treat it as correct except where there is an as-good and more-current source that challenges what it says. Such sources will almost invariably need explicit discussion on article talk pages to assure editors that they are as-good and more-current. For unchallenged assertions, we can simply state them in the former way, while for challenged ones the second would be more appropriate, accompanied by a (cited) statement about the challenge. LeadSongDog come howl! 18:58, 10 December 2012 (UTC)

WP:MEDRS is a specific application of WP:RS which explains how to apply WP:V, a Wikipedia policy. Please read WP:FORUM; if folks would like a fundamental change in Wikipedia policy, perhaps they could take that up on the talk page of WP:V. The essays don't belong here. SandyGeorgia (Talk) 16:59, 10 December 2012 (UTC)

Many strawmen there. Anything in print passes WP:V automatically, and anything that passes WP:V also passes WP:IRS in regard to the limited question of what the source ITSELF says. All V sources are reliable in re what they SAY, and thus pass a limited form of RS. That's a tautology and not interesting. We are after bigger game. And if not at MEDRS, then where to stalk it?

WP has long had an "expert problem" which it has solved by simply saying "go look see what the experts think." Except it's not so easy to tell what the "experts" think, as hardly anybody does formal polls of experts in any field (and if they did, what would their criteria for expertise be?). "High quality secondary/tertiary sources" are the best we can do, but they have many well recognized sources of bias, and the pharm bias in conventional medicine lays on top of the non profit grant-bias that plagues every other arm of the sciences (AM and CAM have the same biases, of course, but the money pool is far smaller). So conventional medical reviews either don't represent expert consensus, or else they actually do, but often cannot be PROVEN to do so (so ^{[citation needed]}). It's only recently that many journals have started to make authors declare their financial conflicts of interest (COIs).

The ONLY question that WP can legitimately ask at WP:RS and MEDRS is whether or not such and such a source genuinely represents come kind of rough consensus of expert opinion (hopefully not too conflicted by money). We actually KNOW that for classificational questions like "is Pluto a planet?" because a poll of experts who had no financial incentive either way, was actually formally done, and reported in print. But that's incredibly rare in science, and usually happens in the formal classification parts of it, when it does happen. Yet, did this happen for any "classification name" you find in DSM-IV? No. And yet, there are gigantic financial COIs in psych diagnosis. That is my basic point. It is a problem that cannot be ignored on WP, either in the DMV or review articles in NEJM, since such reviews are the only "tool" we have for choosing "best" sources.

Finally, when it comes to (quite different matter) of the inductive reliability of expert polls in detecting "objective truth" (limiting this idea of "truth" even to what we'll still think in 30 years after drug/device patents and me-too patents have run out, and a lot more followup data is in)-- that is yet something else again. We should not confuse the two. MEDRS hints that it would like to say something about this last, but doesn't dare to really come out with it. And just as well! It's really unanswerable at the ability level of the average WP editor. In fact the medical profession wonks struggle mightily with it every day, and fail all the time. I can give many examples from recent history, but probably don't need to. Or do I? S B H arris 19:58, 10 December 2012 (UTC)

Another very long essay that says nothing about or suggests any change or improvement to this page or Wikipedia policies (see Colin's WTF post above). Please read WP:FORUM and please confine posts on this page to suggested improvements to this page. If you want to change the way the world turns, or another Wikipedia policy page, this is not the place. If you have a proposal for changing this page, please start a new section and try to state it in fewer words. SandyGeorgia (Talk) 20:06, 10 December 2012 (UTC)

Insufficient context to answer Park, generally we require three things to determine the answer to a 'reliable source' question, 1) The article, 2) The article content, 3) The source (exact part of the source) being used to support the article content. You've really only given us not even half of one of these three requirements--you've named the source book but not a particular page or section within it. Because of this, your question cannot be given an answer.
But, to your general point, I more or less agree with where you're going. It is possible, even likely, that there's some conclusion drawn on some page in the DSM-4 based on 20+-year-old evidence that has since been found to be in error based on studies done in later years. If there's something in the DSM-4 that has since been found to be incorrect according to a WP:MEDRS-compliant secondary source published in later years, we should prefer the article to use the newer information. The DSM-5 is due out in the next few months and then we will look to that as a WP:MEDRS-compliant secondary source.
And, yes, please see WP:NOTAFORUM. Zad68 18:41, 10 December 2012 (UTC)

@Zad, (I am trying, but failing, not to be too wordy) I was trying to avoid specificity of context, and adress that putting our collecive heads in the sand does not make inconsitencies in MEDRS application to DSM go away. Not knowing even of the existence of an "antipsychiatry" movement (until I read about it in the Wiki psychiatry article), and not knowing until today of the history of edit problems with psychiatry related articles due to some editors pushing agendas as gripes due to personal experiences, I was about to try to start and to improve several psychiatry related articles on topics I have familiarity with, including Personality Change Due to a General Medical Condition, Mood disorder otherwise unspecified, General psychiatric disorder, bipolar disorder, autism, and forensic psychiatry. These fomer are "disease" categories said in many sources to be widely abused in forensic psychiatry cases involving involuntary placement and asset seizures, largely due to the vagueness whereby responding to the assertion of their existence is almost impossible under DSM IV or ICD-10 standards. I found the related Wiki articles poorly written, appearing to regurgitate a POV put forward by marketers, not scientists, and largely unsourced. Some of my edits were challenged on grouds that DSM is the "ultimate MEDRS", or some such language. Here is a very specific example off the top of my head, and many more exist in the literature - refusing to give up your wallet and getting hit on the head and knocked out by a mugger, then changing your behavior so that you give your wallet without resistance, is a "disease" by DSM standards, since it meets "Personality Change Due to a General Medical Condition" thresholds. This is not a serious suggestion for an edit, but is to make the point as to deletions of my content on grounds that DSM is the final MEDRS say. I am currently reading History of Bipolar, written by one of the world's foremost experts, who was attacked by the pharmaceutical industry, which tried to make him appear fringe. My using him and his more recent peer reviewed metaanalyses as a source was trumped by the much older DSM IV, according some editors. This leads to the more general question of how to deal with DSM IV as a source, and as estblishing WEIGHT, when it may not. ParkSehJik (talk) 19:43, 10 December 2012 (UTC)

You found autism to be "poorly written, appearing to regurgitate a POV put forward by marketers, not scientists, and largely unsourced". Strange. It's a featured article, quite well sourced, and although I haven't done much there for several months, it was well written the last time I read it. I suggest that you focus on an individual article, engage article talk, and propose changes (attempting some brevity) based on high quality sources; that will be a more productive approach to article improvement, and probably more rewarding for you and for us, than using this page for posting essays. I don't think anyone doubts that 98% of Wikipedia is garbage-- just for reasons different than those you elucidate here. Surely those other articles can be improved, but having a good grasp of Wikipedia sourcing policies is the best way to go about improving them. SandyGeorgia (Talk) 20:11, 10 December 2012 (UTC)

SandyGeorgia just hit Zad68's context specificity nail on the head as to the question here. The Autism article is FA. The first four words are "Autism is a disorder", sourced by DSM, without qualification. There is extensive RS that many adults with autism resent being told they have a "mental disorder". DSM is used as MEDRS that they are "diseased", implying that they would benefit from being "cured", and some are thereby forcibly medicated. Is DSM MEDRS in cases where the category "disease" is recognized to be applied solely on deviation from an arbitary norm, or in cases where DSM is not supported by science? Many Wikipedia articles are written like this, simply pulling a statement out of DSM uncritically, so uncritically that editors pat themselves on the shoulders with the FA award using it. In The Ethics of Autism: Among Them, but Not of Them, Deborah R. Barnbaum obliquely points to many other such problems. So here we have a specific example of the more general question - since DSM provides no information as to which of its diagnoses have a strong evidential basis, which have a weak evidence base, which have none at all, and which are inconsistent with the best evidence, how are we at Wikipedia to know when to use it as MEDRS, and when not to so use it? ParkSehJik (talk) 22:50, 10 December 2012 (UTC)

You have officially exhausted my patience. I have asked you dozens of times to stop using excess markup and hollering at us with bolding. Your assertions are entirely wrong about the autism article, but I'm not spending time reasoning with you on a page where your posts don't belong anyway. SandyGeorgia (Talk) 00:08, 11 December 2012 (UTC)

How are my assertions wrong in the autism article? ParkSehJik (talk) 00:20, 11 December 2012 (UTC) And my last edit summary was "I am putting bold on central question since there is a wall of words by me and others above, but feel free to undo this bolding if it is inappropriate"[34] ParkSehJik (talk) 00:20, 11 December 2012 (UTC)

Park, whether we like it or not, we have to use reliable sources to support claims made in articles. Yes, DSM-4 is 20 years old and may have errors in it, or the psychiatric community may have since changed their position on something found in the DSM-4. As has already been explained to you, this is how we handle it:

If find a newer source of equal or better quality that updates something said in the older source, we use the newer source.
If what the old source says is still current information, and newer sources still say the same thing as the older source, we will keep using the old source if it's particularly authoritative, as DSM-4 is.

DSM-4 calls autism a "disorder." Let's apply the two rules I just mentioned. Do newer reliable sources still call it a disorder? The answer: The NIH, Autism Speaks, this 2009 review article published in The Lancet, the Centers for Disease Control, and the National Institute of Mental Health all characterize autism as a "disorder" and many of those sources still rely on DSM-4 without question. So, YES, in the case of DSM-4, it is indeed a reliable source for information about the characterization of autism. Is this clear? Zad68 04:40, 11 December 2012 (UTC)

Modest proposal

Okay, SandyGeorgia, me use small words. Short sentences. WP editors are not experts, or cannot prove this. Thus core WP policy for natural science must be: WP editors should only cite polls of real experts to find “range of consensus expert opinion(s)” for purpose of determining WP:NPOV, not truth. All other sources (including reviews and texts) are self-evidently those of a small number of article or text authors. They cannot reliably represent consensus in any field, hence cannot be sources of NPOV without WP editors required to weigh different sources and assign relative WP:WEIGHT, which WP editors are by definition not equiped to do and forbidden to do (See WP:SYNTH).

Therefore, suggest erase WP:MEDRS page, replace with: WP:MEDRS consists entirely of finding polls of medical expert opinion, in order to determine NPOV for MED. (Warning: few exist). All other statements in MEDRS about which type of sources are otherwise “reliable,” require ^{[citation needed]} tags, since “reliable” is a guideline of WP which is trumped by NPOV core standard policy. Attempts by editors of MEDRS page to sneakily extend definition of “reliable” guideline beyond simple large expert-poll NPOV policy rule, are themselves unreliable by WP core standard, as these represent editor personal opinion regarding WP:VALID, and are WP-forbidden non-NPOV epistemology. Sorry long word at end, but do not know shorter one.

In short words: if you believe somebody's review or text represents NPOV for a field, HOW do you know this? How can you know it even using many texts or reviews? Are you an expert, or collection of experts? Therefore a ^{[citation needed]} tag is needed for the reliability of your sources as a whole, as you have synthesized relative WP:WEIGHT without benefit of expert poll. Please do not refer to WP:MEDRS guidelines, as it has ^{[citation needed]} tags also. Unless these have been removed by consensus of ordinary WP editors of that guideline. Which is (oddly) how WP normally determines NPOV, not by polling science experts. Kiroc (Unfrozen Caveman Doctor) 22:07, 10 December 2012 (UTC)

No need for snark, SB, it doesn't become you. You well know that the vast majority of assertions on scientific topics have no need of opinion polls. Scientific consensus is amply demonstrated by the publication of an expert review that is not subsequently contradicted in any substantial sense. Systematic review sources (such as Cochrane) make this even simpler by building in periodic checks for currency. We use these precisely because they avoid editor insertion of OR. LeadSongDog come howl! 22:33, 10 December 2012 (UTC)

There's a lot of need for snark here, as most reviews are NOT Cochrane quality. Many "reviews" have a lot of review, and also a lot of "what we do here at our institution." Example right here: I picked up my NEJM for Dec. 6, and found thisnice review on weaning from mechanical ventilation. And was shocked to find the authors recommending 30-minute weaning trials. That's handy, but where did they get that number? I've looked for it. The ICU and pulmonary docs have widely varying practices, and I hear of people who have to survive 30 minutes, an hour, 6 hours, whatever on a T-piece or with no machine support, before anybody will extubate them. It seems to be longer in the evenings, and on weekends. So I looked avidly to see what the science was behind this number (as opposed to some other time), as in the past I've looked myself very hard for it. And I found: nothing. It's not referenced. Citation needed. So to put it bluntly, 30 minutes is only what THESE guys do in their practice (usually) and they are writing it up as a standard-of-care review in the NEJM. It's actually not standard of care (there isn't a standard of care for time in a weaning trial, so far as I can tell). If somebody comes out with some really good science tomorrow to show that 20 or 60 minutes or two or four hours is better for some class of patients, nobody will be shocked. And thus it goes. At the small levels, these review things are synopses of how medicine is practiced today, not synopses of what we actually know and how strongly we know it. Some of that timeless knowledge of physiology is in there, sure, but it's mixed in with "art." Which is fine, but you have to know enough about the subject to know which is which.

Which is why I threw that review across the room in anger, because the authors didn't bother to even point out the controversy and knowledge-gap even as they blithely gave their own uncited answer to it. I guess they have a cite now, hey? They can cite themselves in the NEJM.

Okay, now suppose I'm using this impeccable, unimpeachable source review to edit an article on mechanical ventilation, in WP. Could you rely on me as an ordinary WP editor to know the ice cream from the fruit and chocolate chunks in this review mix, all served up at NEJM and presumably peer-reviewed to the n-th degree? To know that the time was pulled out of some doctor's ... um, hat? The authors haven't helped me. The only reason I noticed this glaring thing was that I was interested in it before, and know something about it already. That's a bit of expertise helping out (though I'm no ICU doc). And we editors on WP do this all the time in the sciences.

The difference is, I know I use my education and my experience. I admit it. In order to write good WP articles and sections of articles I have to synthesize madly, all the time. And use my judgement and background. I would write derivative crap if I didn't (and when I don't, I write crap NOW, or at best, better-edited stuff actually first writen by somebody else, see plagarism). But I admit it. It's against policy, so most other editors not only don't admit it, they don't even acknowledge it. They think they're not doing it (I think). It's amazing. So far will people go to not admit breaking the rules. Which in this case are impossible not to break if you want anything decent that is not plagarized. I'm not the first to point that out, of course. Just venting. Ventolating. I think I may need a weaning trial, and one longer than 30 minutes. Kiroc (Unfrozen Caveman Doctor)

Big concepts not good for folks who only use small words. Over and out-- you are on the wrong page. SandyGeorgia (Talk) 00:10, 11 December 2012 (UTC)

What would be the right page? Kiroc (Unfrozen Caveman Doctor)

Sbharris/Kiroc... what else can be said other than we write articles to be best of our ability using the best available sources we have (and that's on a GOOD day!). The articles will not be perfect and they cannot possibly be better than what our sources tell us. 100% money back guarantee if not satisfied! All I can hope is that experts like you go through and challenge the info and sourcing found in our articles so that we can make them better. Not perfect, but better. Zad68 04:47, 11 December 2012 (UTC)

If you are going to cite sources that were only published this week, you obviously have missed the point above. There is wp:NODEADLINE. Wait a few months and see if that source goes without criticism. If it's truly as outrageous as you think, that will no doubt happen. But we leave it up to the published experts to criticise because pseudonymous editors can't be trusted to know what they're doing, yet wp:anyone can edit.LeadSongDog come howl! 05:33, 11 December 2012 (UTC)

ConsumerLab.com

At WP:RSN#CL we're discussing whether or how ConsumerLab.com's claims about the quality of drugs and dietary supplements can be mentioned in our articles. Experienced MEDRS eyes would be appreciated. --Anthonyhcole (talk) 01:01, 14 December 2012 (UTC)

Should HONCODE certification be accepted as a sufficient measure of quality?

see http://www.hon.ch/HONcode/ What's the benefit in second guessing their efforts?shoi (talk) 23:09, 2 January 2013 (UTC)

Not sufficiently convinced of the reputability of HONcodes. See HONcode#Abuse_of_HONcode Doc James (talk · contribs · email) (if I write on your page reply on mine) 23:22, 2 January 2013 (UTC)

Absolutely not ... I am personally aware of many HonCode sites who are not in compliance with even the bare minimum of HONCODE, but HonCode has no way of making them remove the certification. The whole honcode thing is bogus. SandyGeorgia (Talk) 23:34, 2 January 2013 (UTC)

HONcode certification is not trustworthy. The guidance is too loose, and website owners can manipulate theit information anyway. Axl ¤ [Talk] 12:29, 3 January 2013 (UTC)

Given their mission statement - "promote the effective and reliable use of the new technologies for telemedicine in healthcare around the world" - they should be supporting us. Has anyone here had talks with them? --Anthonyhcole (talk) 14:02, 3 January 2013 (UTC)

Not that I am aware of. Not sure how we would collaborate though. Doc James (talk · contribs · email) (if I write on your page reply on mine) 14:06, 3 January 2013 (UTC)

It's difficult to see how we could benefit from their current strategy, the HONcode. But their commitment should be to their mission, not their strategy, and supporting us would be within the remit of their mission. They are a Swiss not-for-profit foundation, operating out of Geneva, funded by local Geneva authorities. Maybe they could give us some money, advocate for us, support the local chapter. Don't know really - I was just struck by the very close alignment of our missions. --Anthonyhcole (talk) 14:36, 3 January 2013 (UTC)

Problem is, they effect their mission about as well as ... well ... Wikipedia does. Blind leading the blind. SandyGeorgia (Talk) 14:56, 3 January 2013 (UTC)

Mmm. I suspect we're doing a little better than they are. --Anthonyhcole (talk) 15:39, 3 January 2013 (UTC)

If we could get some funding / accommodations to support a Wikipedian in Residence at WHO that would be a great plus. Doc James (talk · contribs · email) (if I write on your page reply on mine) 15:54, 3 January 2013 (UTC)

Great point. What does it cost to support a Wikipedian in residence in Europe with wages and accommodation? Any ideas? --Anthonyhcole (talk) 16:00, 3 January 2013 (UTC)

That Q probably needs an A here. LeadSongDog come howl! 17:09, 3 January 2013 (UTC)

Thanks. So, anything from unpaid, through stipend, scholarship or paid internship to contracted employee. Regardless of remuneration (if any), if we can get free or subsidised accommodation, that would be a big help in finding the best person for the WHO residency. --Anthonyhcole (talk) 07:36, 4 January 2013 (UTC)

Some people are willing to do it for free and cover their own costs. Others are less financially able. So it depends on the candidate. Doc James (talk · contribs · email) (if I write on your page reply on mine) 07:51, 4 January 2013 (UTC)

Primary and Secondary sources

This article implies that we should not cite primary sources at all, or if we do so it should only be temporarily until secondary sources that analize the findings of primary sources (eg. clinical trials) are produced assumedly so that we don't misinterpret the findings of the primary sources such as clinical trials written by those who conduceted them themselves. This a rediculous rule since many clinical trials for example clearly have conclutions by the conductors of the trials that clearlyt explain the outcome of the trial and what it that outcome means. I therefore propose that this rule be ammended indefinitely as in many cases you don't even need a secondary source if the primary source is reliable and the findings are clearly explained. Why shouldn't we be able to quote primary sources ? Even if our interpretation of them is wrong, then that interpretaion can be challenged and then removed, just like secondary sources deemed biased, or unreliable can be challenged, or removed.--197.64.17.242 (talk) 01:25, 26 January 2013 (UTC)

Because we have had too many serious problems with people doing this to promote their pet idea or their employer's new product. WhatamIdoing (talk) 05:06, 27 January 2013 (UTC)

Research comes up with the wrong answer a remarkable number of times. Such papers are not written for a general audience but for the rest of the academic community to interpret, reproduce and consider. The conclusions tend to overstate things as the authors are actually screaming "please publish this", and journals generally only publish interesting results. The unique authorship of wikipedia means we cannot trust authors to be expert enough to judge the research papers directly. Instead, we trust those who write about the research -- secondary sources such as reviews and books. As WhatamIdoing says, many times wikipedians use primary research papers to prove any rubbish theory. However, even authors writing mainstream facts who use primary research as their sources, are often conducting original research, which isn't allowed here. Colin°^Talk 08:31, 27 January 2013 (UTC)

I have made a modest improvement with [35]. The way it was written was actively hostile to documenting biological research, so much so that it would be wiser to start a new project, or better yet to be paid for producing copyrighted articles, than to waste time trying to get in information to articles written for a "general audience" by means of prohibiting all mention of research not formally ready for use in human treatment. Wnt (talk) 17:20, 12 February 2013 (UTC)

I undid the edit... points regarding WP:RECENTISM, COI research, lasting effect and the danger of including bad primary research in an encyclopedia are too great. There is no reason to include it immediately after it is published. If it's good research it'll appear in a review article or meta-analysis soon enough. We should be taking the long view, and wait until the research has been independently reviewed in secondary source before including. Zad68 17:45, 12 February 2013 (UTC)

If what we are trying to do is reflect the current position on a topic than primary sources are never needed. One primary source does not prove anything. We have a user currently attempting to promote colloidal silver using this primary source looking at in vitro work [36]. Only thing worse is the popular presses over interpretation of a primary source [37].

What we need is a stronger and clearer position against primary sources. Even when dozens of secondary sources are present many still do not wish to use them. Doc James (talk · contribs · email) (if I write on your page reply on mine) 20:54, 12 February 2013 (UTC)

WP:Recentism is an essay. A guideline should not defer to it, because it is only a statement of opinion. It's not that it's such a bad opinion, but it isn't all that clear how to apply it, and some people do so in a very detrimental way. Specifically, we should not have Wikipedia be willfully deficient in covering biological research the same way we would cover a singer's career or the deployment of a video game, namely with blow-by-blow analysis by those actually interested, for those actually interested. We should not censor down articles to only such material is of interest to a patient, under the proviso that patients are stupid and patients should not hear anything that might give them hope that useful research is underway.

Fringe viewpoints, as with colloidal silver, can be a problem. I didn't see the precise edit involved, but I can imagine that the paper mentioned could be overinterpreted, or that the Science Daily news story might be itself overly optimistic. But the policy is pushing a fair WP:FRINGE dispute out into "primary" versus "secondary". There is nothing to prevent a review article from having a fringe viewpoint about colloidal silver, or an editor from distorting what a review article says in the same way he might overinterpret a primary source.

I will mention my annoyance this time. I quickly added to Amlexanox a description of a neat result I saw in the news about it having activity against obesity. The article isn't even about a disease, but a chemical, and the point of interest to me was that this chemical had antiinflammatory activity that affected obesity by inhibiting two kinases which I thought might be useful to keep an eye on. Well, meanwhile some editor who believes in this overbearing policy comes along and has a back and forth set of edits with an IP [38] leaving the article under a bazillion complaining tags and with an ominous dated "primary source" tag guaranteeing that when it is 'reasonable' (I don't know if that's five days or five years) they'll be back to delete everything I mentioned. Which is bull. I cited Nature Medicine, for crying out loud. I don't think that this material should be removed at any point, I don't think it needs to be complained about - maybe the result will hold up and I can add, not replace, with a secondary article someday - maybe it will be controverted and I can say at some point what the contradictory results were and cite something that explains them. But the notion that this is somehow inappropriate material, that I shouldn't be adding this when I could instead be working on something that belongs in Wikipedia, like the announcement of the release of the next Pokemon or something, that's just ridiculous. This policy is, in no uncertain terms, telling people that Wikipedia is not to be used to describe biological research, and that if you're doing so you need to stop, wise up, go somewhere else. Wnt (talk) 21:52, 12 February 2013 (UTC)

Yes, WP:RECENTISM is just an essay, but read WP:JUSTANESSAY... although that is, of course, just an essay... :)

It isn't that we want Wikipedia "willfully deficient" in covering research, but rather Wikipedia is an encyclopedia and not a newspaper. I probably should have linked to WP:NOTNEWSPAPER (which is policy) instead of WP:RECENTISM. But they both point to the same idea: As an encyclopedia, WP's articles shouldn't contain information that is questionable, isn't shown to be notable/noteworthy, or likely to change. You even describe the result as something which "might be useful to keep an eye on" - this indicates that you realize that the information on the topic of the use of this drug for obesity is likely to change (if it weren't, why would you have to keep an eye on it?). We shouldn't have information in encyclopedia articles that we need to "keep an eye on".

Releases of most primary study results aren't much more than press releases. In the case of this study of a novel use of amlexanox, the authors describe the result as "emerging evidence" which would indicate very preliminary results need a lot more testing before any level of confidence can be had. This is exactly the kind of thing we should not have in our encyclopedia articles but might be more appropriate at WikiNews.

I took a look at the specific content at Amlexanox. The cite of this study result is a paragraph that takes up more than half the article's prose. Not a single secondary source is cited. What should be in this article is a summary of the three secondary sources available on this drug, PMID 21977491, PMID 21244328, PMID 18335124. The use of this drug as a canker sore med is what the article should be about. Zad68 14:55, 13 February 2013 (UTC)

I have no argument that the article should be longer and should include more information about canker sore treatment. I merely added a paragraph about what was of interest to me. You have to add paragraphs in some order or another, after all.

The "notnews" policy is very commonly misused to mean that "Wikipedia must be out of date". That's not what it says. What it is about is simply treating recent information the same as other information. I left a comment of that type at Talk:Trilantic Capital Partners recently where a PR person had deleted mention of its investment in a company because they had just sold it - clearly, if the article should cover present investments it should cover past investments also. But after a Nature Medicine article like this, nobody is going to forget this drug was described this way in five years or fifty - it's part of its identity in research circles now. So it's not treating breaking news differently to cover that. Wnt (talk) 15:47, 13 February 2013 (UTC)

FYI I have notified WT:MED of this discussion. Zad68 16:01, 13 February 2013 (UTC)

Forgive me but I don't think this response really addressed my point, which is that encyclopedia articles should not contain information you have to "keep an eye on" but rather information that has been shown to have enduring notability (to quote WP:NOTNEWSPAPER). There are articles in newspapers or journals on stuff all the time that doesn't pan out or later proves to have no lasting effect. It isn't that WP must be kept out of date, but rather that it often takes time after something happens to provide the required support for the enduring notability of it. It is the preferred strategy when writing encyclopedia articles to wait until something is really shown to have meant something before including it. In WP-land this generally means waiting for it to get covered in a secondary source. Zad68 16:09, 13 February 2013 (UTC)

To be clear, "keep an eye on" was not meant to suggest I doubted the veracity of the study, but rather, that I expect that after this more research will come out about these kinases which will be interesting to follow. (For example, I was thinking it would be fun to look up and see if there are any GRAS natural compounds that inhibit each of these kinases that could be worked up into a diet potion) My implication in saying that is that as Wikipedia articles are for researchers, not just patients, and so when something seems like eye candy for further research, it is really good to have it in the article. Wnt (talk) 17:40, 13 February 2013 (UTC)

User:wnt, I placed the tags on Amlexanox not to upset anyone or as an excuse to delete content (which I didn't and don't plan to), but to highlight the article for improvement according to policy. The claim that there were inline citation is not strong in my eyes, all the citations are stuck at the end of the article. Do they support the last sentence? paragraph? or the whole article? (in that case may as well be a list of references with no inline). So this is why I didn't really feel that there was acceptable inline citation on the page. I also felt that the other two sources were not desirable, (they were not publications in a peer review journal, or books etc) and could be replaced with better sources. Lesion (talk) 16:26, 13 February 2013 (UTC)

My concern here is more with the guideline itself; how the tags will be interpreted is up to this and whoever comes along. (But I don't understand what you'd expect me to do - cite the same articles at the end of every sentence?) I believe that primary results from a good journal are worth having, and worth backing up with easier to understand secondary-but-not-scientific sources so that less technical readers have a better chance of understanding them. I think it is never a bad thing to add more sources if they increase the sum total of comprehension among all readers following them. Wnt (talk) 17:35, 13 February 2013 (UTC)

You're wrong. Primary sources are only worth having if they are published in a good journal and there are no secondary sources covering the issue and there's a good reason why an encyclopedia should carry that information at this moment. Articles are based on secondary sources, not "backed up" by them. The "easier-to-understand" part is the job of the editor writing the article. --RexxS (talk) 17:57, 13 February 2013 (UTC)

I sometimes find WP:MEDRS to be a pain (mostly due to misinterpretation that articles on medical conditions should only contain medical information) but it's a good policy/guideline. The vast majority of non-randomised primary studies in medicine are not replicated and and even large scale randomised control trials have a fairly high rate of subsequent refutation (10-25%).[39] For medical claims the meta-level research is where the reasonable conclusions lie and that's what this encyclopedia should reflect (although I would tend to cite historically important medical studies that have had a recognised role in reshaping/forming a field or topic of study). FiachraByrne (talk) 23:41, 13 February 2013 (UTC)

... and meta-analyses are, for obvious reasons, secondary sources. FiachraByrne (talk) 23:59, 13 February 2013 (UTC)

Using secondary sources is so easy. Pubmed allows you to restrict your search to just review articles. Often a review that provides an overview of a condition can be sited 20-30 times. This single review [40] could probably get our bipolar article at least half way to GA.

I usually see primary sources being used to promote an idea out of proportion to the literature. There use is often by editors who have come to Wikipedia not to reflect the best available literature but to push a preconceived conclusion and than trying to scrape up literature to support them rather than letting the literature lead. Doc James (talk · contribs · email) (if I write on your page reply on mine) 01:20, 14 February 2013 (UTC)

The search filters on PubMed are imperfect, or rather some publications are listed as review when they are primary sources, and some secondary sources are not marked as review.Lesion (talk) 14:04, 14 February 2013 (UTC)

Wnt, have you considered adding research information to articles like TANK-binding kinase 1 (the enzyme it supposedly affects)? I don't think that it's completely unreasonable to include a brief mention along the lines seen in Amlexanox#Research, but I'm not convinced that details about what's little more than speculation are really appropriate for an article with a clinical focus. WhatamIdoing (talk) 04:50, 14 February 2013 (UTC)

I see that FiachraByrne suggested this overbearing guideline applies only to "medical information", but who knows what will be counted as that and what won't? Amlexanox is a chemical - for all I know you could use it in spray paint. TANK-binding kinase 1 is a component of living organisms. So why would that article be any safer from MEDRS? I would think, of course, that it is not a medical claim to say that a chemical inhibits a protein, but it's been removed all the same, and I see no reason why the same person wouldn't remove it from the other article if I tried. I would think that saying that amlexanox caused weight loss in mice would not be a medical claim either. I don't know if this policy allows a person to discuss traditional uses of plants at all - maybe if you cite Pliny, but not Celsus, since the latter worked more out for himself and is a "primary source"? You should understand that already I have all but abandoned adding this sort of information, this little blurb being a brief exception, and I am feeling like that is the mistake. I could write a blog, post it to Usenet, anything would be more useful than trying to cover this stuff on Wikipedia. Wnt (talk) 15:11, 14 February 2013 (UTC)

Pliny and Celsus are both primary sources. For medical claims, anything more than a century old is going to be used as a primary source. Even an old encyclopedia, although a classic example of a tertiary source, is very likely to be used as a primary source (which is to say, as proof that "A century ago, people thought ____ was the cause of cancer" instead of as "____ causes cancer"). See WP:USINGPRIMARY for a more detailed explanation.

In an article about drugs used on humans, the effects that are seen in laboratory settings (e.g., weight loss in a particular, and particularly odd, strain of mice) are UNDUE unless reported in an independent secondary source. In an article about something else (e.g., an article about the treatment of obesity in mice) it might well be DUE to include information from a primary study in obese mice. But it's not really DUE for that particular article to go into details about speculative research, which is why people keep asking about secondary sources. Relying on secondary sources is a way of proving that something is actually important enough to include. WhatamIdoing (talk) 15:39, 14 February 2013 (UTC)

And you wouldn't say the same about an article about "proteins that occur in humans"? Why not? Wnt (talk) 15:51, 14 February 2013 (UTC)

The main subject of an article on a current, approved pharmaceutical drug is its production and its clinical uses. One lab's speculation about what studies might be interesting to run in the future are not generally appropriate.

The main subject of an article about a naturally occurring protein is whatever people find interesting about it. "This protein is inhibited by (or promoted by) the following chemicals:" is a perfectly reasonable thing to include in an article about the protein.

"This chemical seems to treat obesity in one strain of lab mice on one particular diet" is probably not a reasonable sentence for any article. WhatamIdoing (talk) 19:44, 14 February 2013 (UTC)

Gee, but you realize that the sentence you identify as inappropriate is the one part of what I wrote that was allowed to stand? Meanwhile, is there any basis in this wretched guideline for the distinction you make between a protein and a chemical? If Wikipedia would designate some MEDRS-free spaces maybe they'd be worth working on. Maybe. Wnt (talk) 20:06, 14 February 2013 (UTC)

Honestly if it'd been an article I was coming to cold, I also would have removed the mouse study content entirely "per WP:MEDRS"... because my attention was drawn to the article through this discussion with you, I've left the ref in but cut back the content, summarized the source and put lots of (appropriate) qualifiers on it. Even the source article's summary was written to indicate their research might be of use in the treatment of obesity in humans. What else can be said -- this is a general encyclopedia, there's lots of true, verifiable stuff out there that we don't include in articles, and we have lots of policies and guidelines that tell us what articles we shouldn't create and what content we shouldn't put in. An article on an anti-inflammatory drug use to treat canker sores should overwhelming about its use on canker sores. Information about an organic compound being investigated for its medical effects in an animal with an eye toward its use in treating a widespread disease condition in humans definitely falls under WP:MEDRS. Zad68 20:25, 14 February 2013 (UTC)

I get it, you want to include some of this part: "the noncanonical IκB kinases IKK-ɛ and TANK-binding kinase 1 (TBK1) are induced in liver and fat by NF-κB activation upon high-fat diet feeding and in turn initiate a program of counterinflammation that preserves energy storage" and the research suggests amlexanox inhibits these kinases in a particular strain of obese mice? But without making any sort of connection at all to the idea that it might be useful for treating human obesity? Zad68 20:38, 14 February 2013 (UTC)

Well, that would be a start. I don't think it should stop there because the "not reliable" sources I cited (news articles) gave clear indication that the drug was likely to be tested on human subjects. I of course do not say that is proof of efficacy - to quote [41], "Saltiel is teaming up with clinical-trial specialists at U-M to test whether amlexanox will be useful for treating obesity and diabetes in humans. He is also working with medicinal chemists at U-M to develop a new compound based on the drug that optimizes its formula." To me, that is not some fluff to be ignored, but an actual collaboration with actual funding potential, actual jobs, actual people that should be of interest to any would-be future postdoc who happens to browse the article. Nonetheless, every step forward is a step in the right direction. Wnt (talk) 22:25, 14 February 2013 (UTC)

I should add that your comment that "this is a general encyclopedia" is at the root of the problem. Wikipedia should be an encyclopedia not just for people to read as consumers, as people who buy stuff and gobble pills. Wikipedia should also be an encyclopedia for researchers and workers and administrators, for producers and regulators and such. It should not be a coffee table book to entertain the masses without giving them any real power - it should be a serious effort to make the whole knowledge of the world, including its usable and specialized knowledge, available to whoever cares to learn. Wnt (talk) 22:29, 14 February 2013 (UTC)

The heart of it is that it's an encyclopedia, a tertiary source. Things in an encyclopedia should have been demonstrated to have some importance and lasting effect. This is done by finding citations in reliable secondary sources.

Going back to your specific content, would you be willing to wait until the results of the mouse study showing the drug is a kinase inhibitor have been reflected in some source like PubChem or ChEMBL or the like before including? Zad68 16:58, 15 February 2013 (UTC)

Seriously? You need Nature Medicine to be confirmed by PubChem? I would not think much of PubChem if they missed a source like that, and the more they delay, the more out of date they appear. Which brings up the other huge objection to this review-centrism, which is that when you look at the mechanically-screened list of reviews from PubMed, for every good one there are ten that are not merely in some journal so obscure even a large university library doesn't have access, but which are just not well written, aimed at some very specific point of little real interest, etc. And the best ones are probably written by the same people who published many of primary papers they cover! Wnt (talk) 17:07, 15 February 2013 (UTC)

Wnt, I'm learning something new every day: PubChem it isn't curated, data gets uploaded to it from outside sources, and in this case, the Amlexanox data appears to be coming from DrugBank.ca, and DrugBank is citing individual primary studies in PubMed. According to this article, DrugBank does indeed have a curatorial staff that does fact-checking of what's in the database against the primary sources, but there's no checking that the primary sources themselves are providing good data. The article says of DrugBank, "Even with these added checks and references we still recommend that users carefully study the data sources prior to making decisions about using it." I don't see any evaluation of the quality of the primary source data or criteria to evaluate the evidence and eliminate poor-quality data. So PubChem and DrugBank aren't anything more than repositories of primary source data. So probably the results of the mouse study will end up in PubChem sooner or later but that wouldn't make it a secondary source. And, if I had seen amlexanox listed in PubChem as a kinase inhibitor I would have happily put it in the article. Now that I know how this chain of data works, and I understand it's pretty standard on Wikipedia for pharmaceutical articles to cite PubChem. I no longer have an objection to including the chemical properties of amlexanox as an inhibitor of IκB kinases IKK-ɛ and TANK-binding kinase 1 (TBK1) in the article.

Regarding WP:MEDRS: This is not the first time I've been involved in a discussion with someone on whether the biological activity of a chemical is "biomedical information" and therefore subject to WP:MEDRS or not. Here is what I think needs to happen: We should not weaken the recommendations in WP:MEDRS against primary sources, but instead we need to more clearly define what "biomedical material" means, because that's what the guideline is written to address. We can discuss a proposal to exclude primary study reports of the biological activity of chemicals from the definition of "biomedical material". We should invite WP:MED and whatever the appropriate pharmacology project is to the discussion. If we can come to an agreement on it, I think that would make everyone happy(ier) and we'd end up with better articles. What do you think? Zad68 18:14, 15 February 2013 (UTC)

Well, any progress is progress... I would certainly approve of excluding items from "biomedical material". Still, the text I would prefer for this guideline is more like "A page with this title has previously been deleted..." I would love to feel secure describing the biological activities of chemicals without running afoul of this guideline. I would love to feel secure describing their activities on mice and rats. I would love to feel secure describing what human clinical trials are currently ongoing. The only thing I don't think people should do is to present a recent primary result as established therapy, but that would be misrepresenting the source anyway. But if there's some area we agree on then any progress is progress. Wnt (talk) 23:43, 16 February 2013 (UTC)

I was just made aware of this thread. Interesting discussion. From my perspective, MEDRS should apply only when wikipedia makes an overt health claim along the lines of "X is useful for treating disease Y" or "A causes cancer B." Articles on health-related matters should have clear sections that make such overt health claims, and clear sections about ongoing research. Only the overt health claims should be subject to MEDRS - research is research and things are often being worked out in the primary literature and it is crazy to subject that to MEDRS. Sorry to be US-centric here, but that is where I am from and what I know: I would suggest a division along the lines that the FDA makes - anything that makes an overt claim about treating or diagnosing a disease or condition is subject to the FDA's divisions responsible for drugs and medical devices. So MEDRS should not say "biomedical information" it should say "definitely stated health information - to be more specific, information about health, diseases, medical conditions, and maintenance of health, or actual treatment and diagnosis of diseases and medical conditions." (that would cover things like dietary supplements as well) Further - I completely (!) hear those who have said that MEDRS is useful for getting rid of content about pet or fringe theories. I would also suggest that MEDRS says something like: "Content about research into health, diseases, medical conditions, maintenance of health, and treatment and diagnosis of diseases and medical conditions is not subject to MEDRS, however, such content should be clearly divided from health information, and if there is dispute about the validity or notability of any such research, or undue weight in describing it, MEDRS may be applied to resolve that dispute."Jytdog (talk) 19:28, 17 February 2013 (UTC)

What on earth are you suggesting? Your last quoted sentence makes no sense whatsoever. All these things are subject to MEDRS. And MEDRS is only the application of various policies in the medical/health sphere. The sort of things it recommends, such as avoiding the primary literature is general policy. Can you give an example of some text or article section that you think shouldn't be subject to this guideline but that you think the guideline currently suggests is? -- Colin°^Talk 19:50, 17 February 2013 (UTC)

Ah, I see from Aloe Vera that you think we can have "research" sections that describe all sorts of interesting results with cells, rats and a few people as long as no explicit medical claim is made. Sorry, that doesn't get you off the hook. -- Colin°^Talk 19:58, 17 February 2013 (UTC)

NO. You are almost willfully misreading me. Please read what I wrote above again. And here is what I responded to you with, at aloe vera, copied here for this discussion. "I don't think you understand me. It is important to be super clear about health - definitive statements about human health demand the high standard of MEDRS. No one should make statements about human health based on work in cells or model animals, with the exception of tox studies, and those statements need MEDRS. If someone tries to sneak a health claim based on in vitro or animal research that is not a tox claim with MEDRS sourcing, they should be shot down. Absolutely." Look at my edits on organic food and organic milk and you will see the kind of splitting I mean. Those splits got pages stuck in an edit war unlocked. Look at my edits in the genetic engineering suite of articles, where I have cleaned out tons of ignorant BS about the putative health dangers of GE crops and food. Everything needs limits. MEDRS needs them too. There needs to be a clear and bright line about what MEDRS applies to, and that should be overt health claims only. Pushing it beyond that is silly and just pisses off people on the "other side" (and me too). I know things get hairy and people try to sneak things in. The only way to fight BS is with clarity, not overextending policy. Health claims need to be stated as clearly (not mushily) and supported clearly with MEDRS.Jytdog (talk) 01:14, 18 February 2013 (UTC)

I agree with your stance on the statements that you feel need MEDRS. I just don't buy your argument that requiring a high quality of sourcing must be limited to the small area that you've decided on. "Everything needs limits" may be true but does nothing to indicate where those limits need to be. I want to push tough requirements for sourcing as far through the encyclopedia as I can, and I feel insulted to have that described as "silly". In addition, when I hear someone is "pissed off" because they've been told their sources don't match up to what is needed to validate their pet theory, I don't feel we've done wrong. That is exactly what should be happening - the fringe/pov pushers ought to get that sort of reception in a serious encyclopedia. Finally, I hear your unsupported assertion that about "clear and bright lines" and I thoroughly disagree that MEDRS needs to be ring-fenced in a way that anybody can wiki-lawyer their way round it by claiming that their current obsession isn't strictly an "overt health claim". If we weaken MEDRS, instead of strengthening it, we'll open the door for every snake-oil salesperson on the planet to use our project as their free advertising. No thanks. --RexxS (talk) 02:20, 18 February 2013 (UTC)

Wow, combative. Let's cool down. I hear you that we need to have clear ways to keep out fringe POV content. I am sorry you feel insulted by my use of "silly" - please do not personalize this. But I do think applying policy overbroadly is absurd and worse, is destructive. Take the articles on organic milk and organic food I mentioned above. What was happening in both articles, was that advocates for organic wanted to have all kinds of content about how it is more nutrients and has less toxic stuff, and wanted to draw health claims directly from that. Other editors kept (rightly) shooting them down saying broadly that health claims needed MEDRS. Both sides had good points but they started hating each other and stopped trying to find a workable solution and got into edit wars that led to both of those articles being locked. The solution I proposed and that has stuck so far (although maybe you will go undo it now!), was to create content describing the chemical differences in nutrients and pesticide levels (just standard wikipedia reliable sources) and I put that in a section called "chemical differences", with lots of caveats (sourced) about how it is hard to say much that is definitive about chemical differences between organic and conventional food. It makes sense that this is just standard sourcing -- the content describes the results of benchtop analysis and can be stated without making health claims. And I created a second section about "health" that describes the scientific consensus that there is insufficient evidence for making health claims that organic is better for your health in any way, and why that is the case (sourced with MEDRS). Part of the problem was that neither side was willing to deal with the real scientific problems; part of it was their not thinking through what needed MEDRS and what did not and how they might be separated so that everybody could get what they wanted and we could have a good article under wikipedia's guidelines (the most important thing, right?). So far this has endured - neither side tore it down nor has tried to sneak things in. I am glad that you see that I believe that MEDRS is essential and I am very happy the MEDRS policy was created. I do not agree that having clear criteria for when it applies weakens it in any way. Clear guidelines make policy stronger and brighter and help the community solve problems more smoothly - including the elimination of fringey/POV BS. Maybe most importantly, I disagree with you that "definitive statements about health, diseases, medical conditions, maintenance of health, and treatment and diagnosis of diseases and medical conditions" is "a small area." It is huge. What is your basis for saying that is a small area?Jytdog (talk) 02:49, 18 February 2013 (UTC)

Let's not make any more cheap assumptions about whether we're cool or not. You need to get away from insulting your fellow editors and start engaging on the issues raised - especially accusing me of personalising when you've just done exactly that. I'd quite happily see those 'organic' articles stripped of any content that isn't supportable to decent secondary sources, but I do accept the argument that some encyclopedic content exists which has primary but no secondary sources. Would you agree that it's not the same as saying that any content that has primary but no secondary sources is encyclopedic? The principles encapsulated in MEDRS are not different from the 5 pillars, and have no need to be bounded arbitrarily, no matter how inconvenient they are to editors who write the content first than go searching for references to support it. We have 6,817,134 articles on the English Wikipedia, most of which would benefit from editors adopting the spirit of MEDRS; those that belong to "definitive statements about health, diseases, medical conditions, maintenance of health, and treatment and diagnosis of diseases and medical conditions" represent a rather small number in comparison. --RexxS (talk) 16:13, 19 February 2013 (UTC)

Again, I did not mean to be insulting, it is that just your rhetoric seemed mighty hot to me, and this is a good faith, rational,non-time-pressured discussion about policy, and there is no need for heat here. I don't know what you mean about "artificial" or "arbitrary" boundaries in your edit note and text above. Every policy has boundaries; they are created along with the policy itself. The question is, what should the boundaries be? This is not an irrational nor an artificial question. From my perspective MEDRS is a more strict standard than RS and so is not the same as the five pillars - it goes beyond them. I imagine it was created because the community wanted a more strict policy in this arena, as it did in for BLP. Or do you not agree that MEDRS is more strict? (real question!) Dealing with your questions above, I agree on both counts - there is encyclopedic content in wikipedia that has primary but not secondary sources (although one strives to always use secondary) and there is content in wikipedia with primary sources that is not encyclopedic - assuming by this, that you mean that there is content in wikipedia that should not be there - it was added and had not been caught and fixed yet.Jytdog (talk) 20:00, 19 February 2013 (UTC)

MEDRS is a more strict standard than RS and so is not the same as the five pillars - it goes beyond them - I disagree with this. WP:MEDRS is the application of WP:RS to biomedical information. It does not go beyond WP:RS and it is not more restrictive than WP:RS, it merely clarifies the application of WP:RS to biomedical information. Each general subject area has its own quirks regarding sourcing and what it means for a source to be reliable in relation to it. This is the explanation of how it works for biomedical information, nothing more. Zad68 20:42, 19 February 2013 (UTC)

Can have a "current research" section (in fact it is recommended for diseases articles etc) but that must contain only secondary sources. i.e. reviews of current research (preferably ones that do not make new speculations and hypotheses in their conclusions).... Lesion (talk) 20:14, 17 February 2013 (UTC)

I like this, and I don't like this. Research is research. It deals with emerging knowledge where consensus is not established yet. So.. I agree that where a secondary source is available that summarizes what is emerging (which will probably have several sides and possibilities), that is by far preferable. But content in WIkipedia (that is not making health claims) describing important findings from a primary article published in a high quality journal, where there is no good recent review, seems fine to me -- in a "research" section, not a health section. Here is the tricky part: sometimes research is widely replicated and starts to be used by doctors - it becomes medicine. Then we can have content in a health section about it which will need MEDRS. And it will be MEDRS-able at that point, naturally. Content describing research needs to be stated tentatively, as responsible authors of such works will do naturally. Here is an example of what I mean - check out the edit I did today on the cox-2 inhibitor, celecoxib: http://en.wikipedia.org/w/index.php?title=Celecoxib&curid=648828&diff=538761720&oldid=538755624. It is an interesting case b/c use of cox-2 inhibitors to prevent colorectal cancer is on the edge of becoming medicine, as use of low-dose aspirin to prevent clotting-based heart attacks and strokes has already become. Jytdog (talk) 01:14, 18 February 2013 (UTC)

Ahh, won't it be an unwanted complication to say that part of articles need MEDRS and other parts do not need MEDRS? Also, consider that "health claims" do not need to be clearly stated, they can be implied to the reader with primary sources, which really is what you want to do? Lesion (talk) 01:58, 18 February 2013 (UTC)

One of the good things about the current MEDRS policy statement (and there are many) is that it says "it is vital that the biomedical information in all types of articles be based on reliable, third-party, published sources and accurately reflect current medical knowledge." To me this says the policy applies to "information" - specific content in an article. The "in all types of articles" bit there makes it clear that it is about content, not whole articles. For instance an article about Judah Folkman who is the guy who led the field in figuring out that angiogenesis is really important in cancer (which led to drugs like avastin) would need MEDRS for statements about cancer and how it is treated, but not about where he went to college. I do hear you that people can draw their own inferences from statements about research. There is no stopping that - the most we can do is clearly label content for what it is and if people try to sneak nonMEDRS-supported health claims into content, get rid of them (the content, not the people). We cannot idiot-proof wikipedia. :) Jytdog (talk) 02:21, 18 February 2013 (UTC)

MEDRS already has a limit that covers the case of "where he went to college": a person's educational history is not "biomedical information". The work on the COX2 inhibitor is an example of what MEDRS wants: a proper review article to support claims about what kind of medical research is being done.

(BTW, did you know that you can just type PMID 22928902 to get an automatic link to the abstract, without having to place a full URL in the citation?) WhatamIdoing (talk) 02:54, 18 February 2013 (UTC)

we agree there - i was responding to what Lesion wrote - lesion seemed to be saying that he understands that MEDRS had to apply to the whole article. And thanks for the note about pmid. i like the actual footnote to be compact - its a style thing and as far as i know it is OK to do it like i do it. If i am wrong please let me know.Jytdog (talk) 03:18, 18 February 2013 (UTC)

Assuming that you follow whatever citation style is accepted by consensus at that specific article, you can format it however you want. Most WPMED editors use Diberri's tool to save the trouble of typing anything, but there's no house style that is required at every article. WhatamIdoing (talk) 00:54, 20 February 2013 (UTC)

NPOV and NOR certainly cover implied claims. It doesn't really matter whether you want to directly "say" that ____ cures cancer or whether you want to "just imply" that claim: you need a source that actually makes that claim, and it needs to be a good secondary source.

This means that if you want to "imply" that amlexanox will do something about obesity in humans—say, by making statements that there's evidence to support this belief and that scientists are researching it—then you need a MEDRS-compliant source to back up your claim that amlexanox has some connection to obesity in humans, not speculation by the guy who did the original experiment that it might be cool if he got another grant to do some more research.

I don't agree with Wnt's suggestion above that reporting about this very preliminary research is desirable on the grounds it might help post-docs find jobs. Encyclopedic information is not temporary or ephemeral in nature. "There might be a job here for a post-doc" (assuming that the collaboration pans out, which it might not, and that it gets funding, which it might not, and that they don't already have plenty of people on hand to do the work, which, again, is an uncertain question) isn't really encyclopedic, even if we're "only implying" it. 02:47, 18 February 2013 (UTC)

Not sure who wrote the above, statement is unsigned. Your discussion is getting a bit abstract. Let me try to anchor it. We have an article on obesity - a medical condition. Statements describing it, and how it is diagnosed,a and how it is treated, need MEDRS. There can be a section on research - this section would not need MEDRS but can have no definitive health claims at all. (I don't know what you mean by "imply" - content in this section would be under the header of Research which means "not medicine yet", and the content needs to discuss science, not medicine per se - not how to actually treat or diagnose a disease or condition. The section could have content discussing drugs in clinical trials, drugs being considered for clinical trials, biomarkers that are under study, basic research into what causes it, etc.. Again, with no definitive health statements. If there was a dispute over including some obscure research study, that should be resolved under typical policies (UNDUE, RS, etc) and if that fails, bring in MEDRS. btw, I just looked up the amlexanox article and see that is says "A primary research study released February 2013 of its use in obese mice suggested that the drug may one day be found to be useful in the treatment of obesity." I think this is a pretty reasonable statement - it would be better it were more tentative and said something like "Positive results in a primary research study released February 2013 of its use in obese mice suggests that the drug may be a candidate for further testing in clinical trials in humans" as the abstract says. (actually just made that change) If it said "A primary research study released February 2013 of its use in obese mice showed that amlexanox is useful in the treatment of obesity." this would be completely, 100% unacceptable - that is a health claim and needs MEDRS and the source doesn't satisfy that. Should this be in the research section of an article on obesity? Maybe.. Nature Medicine is one of the best journals. Don't know if there are any review articles discussing amlexanox or its MOA as potentially useful in obesity... WIth that framework, could you maybe explain what you mean by "imply"?.Jytdog (talk) 03:10, 18 February 2013 (UTC)

[42]. But I think I should require $15 to $20 an hour in editing fees to make any effort setting it straight by this point. Wikipedia is only for people who are getting paid to put ads for video games on the main page and such. Wnt (talk) 04:31, 18 February 2013 (UTC)

I hear your frustation!Jytdog (talk) 13:32, 18 February 2013 (UTC)

Consider that to a casual reader, "A primary research study released February 2013 of its use in obese mice showed that amlexanox and may be useful in the treatment of obesity." and "Positive results in a primary research study released February 2013 of its use in obese mice suggests that the drug may be a candidate for further testing in clinical trials in humans" will give similar meanings. A previous discussion on WT:MED concluded that it is acceptable to use primary studies in the "history" section only, e.g. landmark historical papers. Now here you are suggesting that the "research directions" section may also contain primary sources, which I do not think reflects the consensus. I think most people would say the research directions section must also contain secondary sources and no primary sources. If a primary study is printed in a good quality journal is less important to the fact that it is still a primary source imo. When I write a medical article, first I look for a cochrane review, then textbooks and other reviews. It could be argued that any info not covered in those kind of sources is not notable. If I find a primary study with content I want to include, I look for reviews which cite it. Furthermore, I think the content should reflect the amount of coverage given to the primary study in the review, to avoid undue weight. I see the logic of MEDRS now, e.g. to avoid articles like this. Note instances of several primary studies probably inserted at the same time by a user with a strong belief in a single hypothesis over the many others, creating undue weight in the article. We have a load of primary research papers saying that zinc deficiency or some chemical is responsible, and now looking through good quality secondary sources I find either no mention or a single sentence mentioning these factors. I have even had to go back over (some) of my early articles and replacing all the primary sourced content with secondary sources, which is a huge pain but improves things. Lesion (talk) 04:26, 18 February 2013 (UTC)

The "Causes and epidemiology" section of the canker sore article is indeed a mess. This is health information and I agree 100% that it needs MEDRS.Jytdog (talk) 13:32, 18 February 2013 (UTC)

Jytdog, please restrain your edits to short to-the-point responses. All WP:MEDRS is the application of various policies to a subject domain. The underlying policies like WP:WEIGHT apply everywhere. You can't fill an article with a random collection of primary research findings based on your reading of the primary literature -- it completely fails WP:WEIGHT and is impossible to do without implying something to the reader -- that these findings are important and deserving of your attention. Who says they are important and deserving of the reader's attention? Not Jytdog, not me, not any Wikipedian. You need secondary sources. Basically you are asking us to sanction your writing of a review of the primary research literature. That's WP:OR. Colin°^Talk 10:11, 18 February 2013 (UTC)

You haven't explained what you mean by "imply" - I am trying to say that all we can do is control what the content actually says; we cannot control what readers do with it. You seem to be assuming bad faith by me in contravention of [WP:AGF]] - I think the reason for that policy is what is happening here - you seem to be too busy fighting things you are worried about to deal with what I am actually saying. I would never fill an article with a random collection of primary articles, nor am I trying to establish a policy where people could do. Even RS says that secondary sources should be used first. I am not arguing with that.Jytdog (talk) 13:32, 18 February 2013 (UTC)

I'm absolutely not assuming bad faith. The "you" in my wording is plural and often hypothetical. Your opening remarks talk about "Content about research into health, diseases, medical conditions, maintenance of health, and treatment and diagnosis of diseases and medical conditions" not being subject to MEDRS. It is simply impossible to mention basic research into health and medical issues without implying it has clinical significance. And it is impossible to explicitly state it does or does not or might or might not have clinicial significance without a secondary source. The mere mention of such basic research means that it has WP:WEIGHT. For example, mentioning that X causes cancer in rats, where X is a human foodstuff or cosmetic, etc (and not some industrial chemical, say). Lots of things, including much that we happily eat or cover ourselves with are toxic to rats at some level of exposure. Why would any encyclopaedia article even mention this "naked fact" other than to imply there might be a human danger. Ultimately, you haven't made it clear what parts of MEDRS are problematic here, but rather seek to remove it from some of the most problematic areas of articles: health research. Why? Colin°^Talk 14:36, 18 February 2013 (UTC)

Colin could you also comment specifically on the other part of this conversation here? The thread from "I get it..." through "Well, any progress is progress..." Thanks... Zad68 14:58, 18 February 2013 (UTC)

Out of time today, I'm afraid. -- Colin°^Talk 16:50, 18 February 2013 (UTC)

There is no difference between skewing an article by cherry-picking the primary sources you want and skewing an article by cherry-picking the secondary sources you want. The latter is probably easier because secondary sources have built-in biases (they want to say what will treat obesity, or that their company's product is useful, etc.) while primary sources are usually more of a "here, I tried this, and this is what I got" kind of result. The only acceptable way to avoid bias from either is to let people cite them all, grouping redundant sources either by omitting the less interesting ones and citing reviews that cover them as a group, or branching off minor but well-documented points into sub-articles of their own. The ideal result is something that reads like an OMIM entry. Wnt (talk) 16:08, 18 February 2013 (UTC)

I don't know where you get the idea that secondary sources are more biased than primary sources. Primary research papers aren't "here, I tried this, this is what I got". If from a drug company they are "here, this stuff works, please FDA give us a licence to make money" and if they are independent then they are "here, we've found something we think is interesting and speculate is of vast importance to mankind, please give us grant money so we can keep our jobs and research some more". And you never read the primary research paper than says "here, we found this doesn't work" or "here, we found bugger-all of any statistical relevance". There are biases everywhere. The "let people cite them all" or "just include all the sides and let the reader make up their own mind" are cop-outs and just an escuse to include "facts" that have no WP:WEIGHT. We are called editors for a reason. Colin°^Talk 16:50, 18 February 2013 (UTC) -- We are an encyclopaedia, not a database for medical researchers nor are we literature review for physicians. The sort thinking going on here forgets (a) our audience and (b) our unique and odd lack of trust in anything an editor might think or say for themselves. -- Colin°^Talk 16:54, 18 February 2013 (UTC)

I couldn't disagree more. This is the absolute root of the problem. NPOV is not a cop-out, it is supposed to be a fundamental principle of Wikipedia. Covering a topic completely is not a bad thing, it should be our goal. We should not be choosing our audience, video game purchasers ahead of AP biology students looking for an advanced thesis topic, but appealing to every audience. Wnt (talk) 17:11, 18 February 2013 (UTC)

You're completely wrong. If you think that WP:NPOV means that all sources should be included, then you need to check out what we mean by 'proportionate', 'unbiased' and 'significant'. Then see if you can understand why we don't let editors be the final judges of those principles. The ability of secondary sources to make those judgements for us may not work perfectly, but it works a lot better than allowing you or me to decide whether a particular primary source is worthy of inclusion. --RexxS (talk) 15:35, 19 February 2013 (UTC)

As I said above, the same selection bias can be imposed by the choice of secondary sources as through the choice of primary sources. If somebody wants to exclude information they think is "fringe science", let them at least show that a review on the topic actually pointedly ignores a concept, despite having access to it, or actually dismisses it as false. Let them argue that it is a really obscure journal that publishes anything. The overwhelming assumption that any biologist would make is that a new primary source in a halfway decent journal has something important to say! Wnt (talk) 21:30, 19 February 2013 (UTC)

Agree with RexxS. Being comprehensive is not the same thing as including all possible facts and research findings in the pot and hoping the reader likes the soup. WP:NPOV says "Keep in mind that, in determining proper weight, we consider a viewpoint's prevalence in reliable sources, not its prevalence among Wikipedia editors or the general public." and those sources are secondary ones. A primary research paper cannot determine its own WP:WEIGHT no matter how hard the authors may try. It seems quite clear that Wnt and Jytdog wish for a different kind of Wikipedia from the one we have. Colin°^Talk 21:25, 19 February 2013 (UTC)

I do not agree that the wikipedia we have today interprets MEDRS the way you do, Colin. The breadth of your claim - that everybody interprets MEDRS like you - is somewhat stunning.Jytdog (talk) 22:09, 19 February 2013 (UTC)

If people don't interpret MEDRS like Colin does, then they're probably wrong, because nearly all of MEDRS was written by Eubulides, Colin, and me. WhatamIdoing (talk) 01:02, 20 February 2013 (UTC)

Wow that is great! Thanks for having done that. It does explain Colin's attitude a bit - I reckon there is endless nonsense to knock down but some WP:TRUTH seems to have crept in.Jytdog (talk) 01:30, 20 February 2013 (UTC)

From the one you've interpreted, and devastated with this overbearing, fundamentalist zeal, that is. Wnt (talk) 21:32, 19 February 2013 (UTC)

From this "WP:Weight" thing you go on and on about: Neutrality requires that each article or other page in the mainspace fairly represents all significant viewpoints that have been published by reliable sources, in proportion to the prominence of each viewpoint in the published, reliable sources. Could it be clearer? Is Nature Medicine not a reliable source? Is a major research article in it not significant? Wnt (talk) 21:36, 19 February 2013 (UTC)

Reliable sources are secondary sources. It is my understanding that this is wikipedia wide policy not just MEDRS. Again, my interpretation, but the quality of the journal is a less important factor to whether it is a primary or secondary source. Lesion (talk) 22:22, 19 February 2013 (UTC)

From WP:RS: Primary sources are often difficult to use appropriately. While they can be both reliable and useful in certain situations, they must be used with caution in order to avoid original research. This could be clearer, but it is clear enough! Wnt (talk) 22:55, 19 February 2013 (UTC)

@Wnt: Regarding the second question: it may well be that it is NOT significant. There are many articles every year in very reputable journals that are never again cited, have zero impact in science and in summary are not significant in any way. Best way to know if they are significant is secondary sources, not you or me deciding here which ones had a greater impact or not. Therefore, regarding your first question: no, nature medicine is probably not by itself a reliable source for wikipedia. What is a reliable source is a secondary article published in Nature medicine. Even worse: there are cases were a primary article is simply wrong, misleading or unreplicable. In this cases it is much less probable that a secondary article falls in the same problems that the primary one. Have you heard of Andrew Wakefield?. He surely published in lancet a primary article which had a great impact... I do not think however that there are many reviews supporting his claims...--Garrondo (talk) 22:33, 19 February 2013 (UTC)

Look at MMR vaccine - search for Wakefield. Yes - it references his primary study. In fact, it references a retracted paper because that is what is necessary to edify the public who goes into that article having actually heard all that publicity, regardless of whether it was true or false, and who need to be educated. Now if it is appropriate to reference that paper now, when it is utterly discredited, how much more appropriate it was to reference it when it was merely hotly contested! Wnt (talk) 22:59, 19 February 2013 (UTC)

Breaking up (is hard to do)

Sure, we cite Wakefield's paper. It's historically important. That doesn't mean that we should cite this paper.

There are two major related issues being discussed: one is DUE weight, and the other is writing so readers understand. Let's talk about them separately:

Garden-variety due weight issues: This year, there will be more than ten thousand publications about obesity in peer-reviewed, MEDLINE-indexed academic journals. What's so special about this one, compared to the other >100,000 articles published in the last decade? Compared to the >100 articles published just in Nature Medicine on this subject? Well, probably nothing, except that it's interesting to a couple of Wikipedians.
What our readers understand: A few of our readers are post docs. Most of our readers aren't. Most of our readers are people who honestly do not understand that there is an enormous gap between "A scientist researched this" and "It works". One of the reasons that we exclude speculation like this is because we want to educate our readers, rather than feeding them things that we know nearly all of them will misunderstand.
It might be easier to use a different example. Forget about amlexanox. Think about Alternative cancer treatment. You want to add "A recent study in mice reported that amlexanox may have activity against obesity and diabetes, where it appears to work by increasing basal metabolism... Because of its record of safety in patients, amlexanox may be an interesting candidate for clinical evaluation in the treatment of obesity and related disorders". A lot of snake-oil salesmen would dearly love to add "A recent study in mice reported that Juice O' the Day may have activity against cancer, where it appears to work by increasing basal metabolism. Because of its record of safety in patients, this juice may be an interesting candidate for clinical evaluation in the treatment of cancer", and they want to cite equally limited preliminary research results. Do you think their additions would be valuable? What's the measurable, objective difference between the speculative research you want to add and the speculative research they want to add? WhatamIdoing (talk) 02:41, 20 February 2013 (UTC)

There reason why we can handle 100,000 papers about obesity is that we can divide them up among thousands of articles. A paper about amlexanox belongs under amlexanox - a paper about dinitrophenol belongs in that article. (To be frank, I have no illusions about this drug - I know full well that the odds are it will do what most anti-obesity drugs do, namely kill some people sometime either shortly before or shortly after it is slated for government approval. But hope springs eternal...) There are indeed situations where the total number of citable primary articles for a given article would be so high that some extra effort is needed to organize them, such as by topic of research or along other conceptual lines or by citing secondary articles to cover many of the primaries. For big topics like obesity there are of course far too many review articles to cite them all in one article also. Wnt (talk) 04:07, 20 February 2013 (UTC)

Looking at alternative cancer treatment, it looks like the arbitrary decisions by editors are still potentially made - for example, who decided if Budwig diet is on the list (which it is now)? But if we resolve to allow every treatment to be mentioned, and divide up the article if need be to accommodate them all, then we maintain NPOV, and provide a comprehensive resource of bizarre notions for cancer treatment. Wnt (talk) 04:12, 20 February 2013 (UTC)

Wikipedia is not a comprehensive database of scientific publications. We are not supposed to "accommodate them all". We don't want to divide 100K of publications across our 200 obesity-related (or semi-related) Wikipedia articles. Having five hundred primary sources (on average) in each obesity-related article is proof that someone has screwed up, not proof that someone has achieved NPOV by including every possible source. WhatamIdoing (talk) 17:55, 20 February 2013 (UTC) WhatamIdoing (talk) 17:55, 20 February 2013 (UTC)

If we actually documented the field well enough to cover those 100K papers, there wouldn't be 200 obesity-related articles, there would be thousands! True, expanding Wikipedia by 20-fold would require a little more technical sophistication in places - how people search for articles, how articles direct readers to one another, how reference sets are organized and so forth - but it's not something that will happen overnight and there is plenty of time for technical improvement. Right now it doesn't look like it will ever be done, because Wikipedia doesn't deserve the volunteer effort. Wnt (talk) 19:09, 20 February 2013 (UTC)

Note

Just a quick note. I am withdrawing from this. I carefully re-read RS and MEDRS and I think it allows flexibility to use primary sources where they are really needed and can be justified, and not otherwise. What I really care about is having a clear scope for MEDRS so I have opened a new section to see if we can get consensus to fix it. Colin expresses himself in a difficult way but I actually agree with most of what he says and have argued for a broad definition of MEDRS.Jytdog (talk) 23:30, 19 February 2013 (UTC)

Problems with Secondary Sources

Secondary metastudies ('Wikipedia gold' as it has been described to me) are not properly vetted by the peer review process most of the time: http://www.cochrane.org/news/blog/how-well-do-meta-analyses-disclose-conflicts-interests-underlying-research-studies . However, Wikipedia policy does not allow editors to reject these studies due to COI or other flaws. The result is that these studies may become part of Wikipedia without any effective vetting process. This is wrong. A large number of secondary/metastudies are funded by special interests. And since editors aren't allowed to reject studies based on COI, this ensures that special interest viewpoints are part of the articles. Primary medical studies which contain actual results and, when used as an overall preponderance of evidence represent scientific fact but are not allowed. The result is many Wikipedia articles do not represent any sort of scientific consensus OR fact. I believe that this must change. Crimsoncorvid (talk) 04:29, 13 February 2013 (UTC)

I ran into a case like this a few months ago on homeopathy. My position on the topic is that a meta-analysis is an experiment, not a review, and therefore a primary source - it takes a set of data and uses an algorithm to generate a result, rather than presenting all the data and reconciling it by logical deduction. The meta-analysis goes beyond what someone could do in a simple review, because it is willing to strike an average rather than declare a consensus. But then again, I support broader use of primary sources than some here (see above). Wnt (talk) 05:47, 13 February 2013 (UTC)

I read that pharmaceutical companies fund all research and reviews up to the point when a drug hits the market. After that point, they fund 60% of the cost of reviews. So, in a sense, most pharmaceutical reviews are funded by special interests. TimidGuy (talk) 16:16, 13 February 2013 (UTC)

Although the problems of undisclosed CoI may weaken the value of a particular secondary source, they actually derive not from undisclosed CoI on the part of the reviewers, but from undisclosed CoI in the primary studies:

"A recent study published in JAMA ... found that conflicts of interests in the studies underlying the meta-analyses were rarely disclosed." - opening sentence of http://www.cochrane.org/news/blog/how-well-do-meta-analyses-disclose-conflicts-interests-underlying-research-studies

It just goes to show how problematical primary sources are. At least a meta-analysis gives the reviewer a chance to do the research and balance what they know about any CoI undisclosed in the primary studies. Is the purpose of pointing this out to suggest that we grant Wikipedia editors the right to do that meta-analysis themselves? --RexxS (talk) 17:48, 13 February 2013 (UTC)

The review published in JAMA was PMID 21386079. It doesn't really support Crimsoncorvid's summary above, nor does the blog post that he cites. The same authors published followup research at PMID 22906823. The issue they discuss is whether Cochrane reviews are consistently doing a good enough job of reporting and considering any COI in the primary studies being reviewed. In effect, they found that while the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) already requires the authors of the reviews to disclose their own COIs, but need tweaking to include consideration of COI that might affect included primary studies. That is a very far cry from saying that the reviews are less objective then the primary studies.LeadSongDog come howl! 18:45, 13 February 2013 (UTC)

Indeed... I'm also scratching my head over the logic that because independent secondary sources may not be doing a perfect job of identifying the COI of the primary studies then therefore the problem can be solved by: citing the primary studies. Zad68 18:52, 13 February 2013 (UTC)

IMHO, some of the best sources for articles about scientific topics, including many medical topics, are peer-reviewed reviews of the literature, including both stand-alone review papers and the literature reviews included in papers describing new research. Literature reviews have the benefit of being both peer-reviewed and secondary sources. --Orlady (talk) 04:33, 14 February 2013 (UTC)

Crimsoncorvid's edits are largely about Circumcision. That's a major topic for which only secondary sources should be cited throughout, because there are hundreds or even a few thousand of them. It is also exactly the kind of topic for which our prohibition on rejecting sources because of an editor's beliefs about funding or "conflicts of interest" was written: "Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions." When controversial subjects like this one are being discussed, almost every objection to the "funding sources" turns out to be an editor looking for a "polite" or "policy-based" way to disguise WP:POV pushing.

So to answer Zad's implicit question: the way you solve the problem of independent secondary sources failing to directly list all of the COIs is by using only primary sources that give the right answer. The way you know that these primary sources give the right answer is that they happen to agree with the editor's personal viewpoint on the subject. WhatamIdoing (talk) 22:01, 13 February 2013 (UTC)

I am not POV pushing. The Circumcision article is already laden with pro-circumcision propaganda. So when the AAP, which has a vested interest in promoting circumcision to support its members publishes a meta-analysis or review, you don't think that has a COI problem? You are missing my point entirely. What I'm talking about is denying FACTS from the article that come from primary studies. The secondary studies on many topics, including circumcision, are largely funded by organizations that are trying to promote something. The result of the Wikipedia policy has nothing to do with editors pushing a POV and has everything to do with including sources funded by special interest. The result is that articles become no longer representative of scientific facts. Crimsoncorvid (talk) 03:52, 14 February 2013 (UTC)

I understand that the Cochrane link I sent was about failing to find COI in primary studies. This is a problem too. But the problem also exists for secondary studies which have no external review. Wikipedia has a big problem of holding certain sources (WHO, AAP, etc.) as INFALLIBLE. This is clearly wrong as they too have their motivations. I believe that studies must be vetted. If you think that these policies are exactly to prevent controversial articles like circumcision from editor POV, you're living in a dream world. Not only are these policies causing the Circumcision article to be biased against scientific and medical consensus, but it is giving the pro-circumcision advocates here on Wikipedia a launching off point to bias the article. And if you don't think that article's biased, you either don't understand the circumcision issue and all its facets (scientific evidence, ethical evidence, etc.) or you're a circumcision promoter yourself. Crimsoncorvid (talk) 04:16, 14 February 2013 (UTC)

For those wondering what's going on here, I believe this is the issue Crimson is talking about: Right now the circumcision article has a statement cited to a recent AAP position statement and its 30 page technical report, two different review articles, and a systematic review; these sources used data from (among other sources) two large (N=5000 and N=2784) 2008 RCTs that were designed to measure the effects of circumcision on sexual function and satisfaction, and penile sensitivity, and all four of these secondary sources agree in their reporting on this subject. Crimson would like to counter this with the results of two primary research studies: PMID 23374102, a non-randomized online self-assessment survey, N=1059, subjects were self-selected through recruiting advertising; and PMID 17378847, N=159, a study of non-randomized participants, some recruited through fliers, some invited by the researchers directly. Zad68 05:53, 14 February 2013 (UTC)

Given what is very clearly going on at the circumcision article and the likely investment of time required to prevent inappropriate/unencyclopedic material from being included, would it not make sense to seek a topic ban? If such a topic ban was sought what would be the likelihood of success? These kinds of problems seem endemic to much of the encyclopedia. FiachraByrne (talk) 10:51, 14 February 2013 (UTC)

It's a consideration, I'll be sure to notify if something happens... Zad68 13:39, 14 February 2013 (UTC)

Zad68: You are completely incorrect. The studies you refer to (besides the AAP one) are studying affects of circumcision and STDs. There were not studying penile sensitivity directly. All of these studies have COI issues (especially the AAP one). You can ban me all you want, but that doesn't change scientific and medical fact. Crimsoncorvid (talk) 15:13, 14 February 2013 (UTC)

Look, it's simple: Nobody here can judge whether a particular source is INFALLIBLE - we can only work out whether it's RELIABLE. You don't get to decide that major position statements from well-respected organisations are CoI; you'll need a good quality source to back up that accusation. The vetting that you desire for secondary sources is done by peer-review and by the reputation of the editorial process of the journal where it is published, and we're not going to hand that process over to editors with their amateur analyses. --RexxS (talk) 15:48, 14 February 2013 (UTC)

I appreciate your opinion and I do see your point about not wanting to hand the process to "amateurs". But are the "professionals" doing a good job either? I feel the need to set the record straight. Zad68 wants you to believe that I'm POV pushing and that my sources don't hold up to the reliable sources in the article. There are 43 sources discussing penile sensitivity opposed to the five opposing secondary soruces quoted in the article. For convenience, I'll link you here (look at the sources): http://intactipedia.org/index.php?title=Foreskin_Sensitivity . The "reliable secondary sources" used in Circumcision DO NOT represent science which is what I care about. Crimsoncorvid (talk) 12:30, 15 February 2013 (UTC)

It isn't just that editors here are amateurs (some are not). The point is that because there is zero accountability for anonymous editors we cannot expect readers to trust their judgement. Indeed it would be purest folly to do so. Instead, we say that readers are entitled to know that a specific published secondary source backs up the analysis. They can read that source for themselves and see their report of how they selected the primary sources to use. The authors, peer reviewers and publishers of the secondary sources provide accountability that anonymous WP editors simply cannot. Wikipedia's verifiability policy is not negotiable. It doesn't matter that you don't like it. Go find reliable secondary sources and then we can talk. LeadSongDog come howl! 16:38, 15 February 2013 (UTC)

I think any usefulness of this discussion to proposing improvements to Wikipedia's WP:MEDRS guideline has now been exhausted. It seems pretty clear that there's no consensus forming for any changes to the guideline. Crimson has brought the link to the list of sources to Talk:Circumcision, the appropriate page to discuss article content and sources, and that content discussion should continue there, not here. This discussion should be closed. Zad68 16:46, 15 February 2013 (UTC)

To the contrary, the OP's point about meta-analyses remains valid, and perhaps it would be best to RfC it to get more input. There are a lot of issues here that could be raised in a combined RfC. Wnt (talk) 17:12, 15 February 2013 (UTC)

It's an important point to be mindful of, but I'm not sure how it should be used to change the existing wording (or if the original poster had any specific example they could point to that is flawed). A specific example should be demonstrated, for the purposes of discussion, in my opinion. Biosthmors (talk) 20:13, 15 February 2013 (UTC)

The OP's point about meta-analyses has been shown to be invalid. The blog cited and the original JAMA paper do not support his conclusion that meta-analyses "are not properly vetted by the peer review process most of the time" or the opinion that letting editors pick and choose those primary research papers they think document the true science rather than the bias is somehow an improvement. I don't think there is any merit so far in a RfC as there is nothing offered worth wasting people's time discussing. Colin°^Talk 10:38, 17 February 2013 (UTC)

Proposed change to opening words

MEDRS starts out saying: "Wikipedia's articles, while not intended to provide medical advice, are nonetheless an important and widely used source of health information.[1] Therefore, it is vital that the biomedical information in all types of articles be based on reliable, third-party, published sources and accurately reflect current medical knowledge....This guideline supports the general sourcing policy at Wikipedia:Verifiability with specific attention given to sources appropriate for the medical and health-related content in any type of article, including alternative medicine. Sources for all other types of content—including all non-medical information in medicine-related articles—are covered by the general guideline on identifying reliable sources rather than this specific guideline.

This is somewhat confusing. Is "biomedical information" meant to be the same as "health information" and as "medical and health-related content" or do all these mean different things? How do they relate to "current medical knowledge"? If they are different from one another, how? Guidelines/policy should be very clear about their application, and I have seen endless arguments over the meaning of "biomedical information" etc. If the policy were more clear about what it covers, then there would be fewer disputes and they would be more quickly resolved.

So let's define the scope - the type of content subject to MEDRS, give it a short name, and use that short name throughout MEDRS.

Some options, starting with clearest focus on human medicine (MEDRS) and broadening out to include any animal and human medicine, then to include research into health and medicine.
1) content about human health, human diseases and medical conditions (including their description, incidence, prevalence, causes, and prognoses), maintenance or improvement of human health, and actual treatment and diagnosis of human diseases and medical conditions, as well as behaviors, devices, chemicals, and organisms that harm or aid human health ("medical content")
2) content about health, diseases and medical conditions (including their description, incidence, prevalence, causes, and prognoses), maintenance or improvement of health, and actual treatment and diagnosis of diseases and medical conditions, as well as behaviors, devices, chemicals, and organisms that harm or aid health ("medical content") (note this one includes animal health and medicine)
3)content about health, diseases and medical conditions (including content about their description, incidence, prevalence, causes, and prognoses), maintenance or improvement of health, and treatment and diagnosis of diseases and medical conditions, as well as behaviors, devices, chemicals, and organisms that harm or aid health, including basic and clinical scientific research concerning those topics. ("biomedical content")

There you go. I favor the broadest, (3) , for being most useful for screening out junk.Jytdog (talk) 23:24, 19 February 2013 (UTC)

How about:

4) content that represents itself as describing currently accepted standard practice in the diagnosis, treatment, and prevention of medical conditions.

Notably excluding alternative practice (do you expect a MEDRS to list and describe the acupuncture meridians?), traditional medicine (Hippocrates' techniques), and of course, ongoing biological research. Wnt (talk) 00:08, 20 February 2013 (UTC)

I don't like 4). Leaves out description of the disease itself. "represents itself as" is too meta. "MEDRS" does not list anything - it is a guideline for sourcing content. I thought about practices seen as alternative in the west like acupuncture. Current guidlines says "This guideline supports the general sourcing policy at Wikipedia:Verifiability with specific attention given to sources appropriate for the medical and health-related content in any type of article, including alternative medicine." And I agree with that. I am just trying to get an improvement to the "subject matter" - in this sentence, called "medical and health-related content".Jytdog (talk) 00:41, 20 February 2013 (UTC)

Leaving CAM out of the equation is a mistake. Anyone will be able to say anything about it. Perhaps I'm mistaken; MEDRS would still apply to CAM, no? DVMt (talk) 03:07, 20 February 2013 (UTC)

I can see 4) being the kind of thing proposed by someone who disagrees with the existence of the WP:MEDRS guideline entirely.

The things for which we most desperately need clear guidance to use well-respected independent reliable secondary sources would be entirely omitted from WP:MEDRS scope. The (unintended?) consequence could be something like, to use the obesity example cited above, the possibility that our obesity article could be stuffed with any or all of the results of the 4,876 in vitro and clinical trials on obesity published in PubMed in the past 10 years, including things like:

PMID 23168347 "A true challenge for any superhero: an evaluation of a comic book obesity prevention program"
PMID 21303505 "Short-term outcomes of community-based adolescent weight management: The Loozit® Study" phase 1 clinical trial in 150 kids, article content could be based on "The Loozit(®) program may be a promising option for stabilizing overweight"
PMID 16783408 "Chronic 5-HT6 receptor modulation by E-6837 induces hypophagia and sustained weight loss in diet-induced obese rats", a 2006 primary study, article content could be based on "the 5-HT(6) receptor partial agonist, E-6837, is a promising new approach to the management of obesity with the potential to produce greater sustained weight loss than sibutramine" (number of subsequent articles since 2006 that mention E-6837 and obesity: zero, but whatever, go ahead and put it in the obesity article)

so I'm not seeing 4) as leading to a better encyclopedia. Zad68 04:09, 20 February 2013 (UTC)

I should note that I didn't put amlexanox into the obesity article, as obviously there are so many possibilities to shove in there - though I would support branching off an article like List of substances showing anti-obesity effects in animal studies (perhaps further sectioned into subtopics based on classes of biological activity) and having both compounds as entries in it. Wnt (talk) 04:16, 20 February 2013 (UTC)

Understood and appreciated. What if someone else found the same primary study but was not as judicious as you were? Can we word the suggested scope definition to cover it? Maybe if you can explain why you didn't think to add it to obesity that would be useful to craft this. Zad68 04:29, 20 February 2013 (UTC)

Well, like I said, I'd see it as a very minor point relative to that article. In theory, it is better to have even a very minor point like that added to a main article with the idea that someone else will come along behind and start the list article I suggested; but in practice I know too well that either it or some other fact from the article would likely get squeezed out without the needed structural revamp, so I wouldn't bother unless I were ready to collect enough items to start the list article myself. And under constant fire from this policy I have no intention of such an ambitious project, not anywhere on Wikipedia at least. Wnt (talk) 04:35, 20 February 2013 (UTC)

CAM and alternative medicine absolutely need to be covered. Not necessarily the theory behind how it might work, but the results. If there is a claim that acupuncture on "meridians" has the effect of lowering blood pressure or whatever, yes it needs to be covered. The "meridian" theory might be sourced to some authoritative text, but honestly if it something to do with human health it's probably in PubMed, and if it isn't in PubMed it's probably not worth looking at. In fact for starters see PMID 22322253 "A review of Omics research in acupuncture: the relevance and future prospects for understanding the nature of meridians and acupoints." I see at least 100 review articles from the past five years covering accupuncture in PubMed. PubMed has huge scope. Regardless, any claim that can be stated as "Intervention X has health effect Y" needs to be covered by the guideline. Zad68 04:21, 20 February 2013 (UTC)

(ec) I can find some interesting reviews out of a few of the 22000 hits for obesity reviews also.

[43] "There were fewer reports of adverse effects and relapses of weight regain in CHM intervention studies conducted in China than studies conducted outside China. CHM and acupuncture were more effective than placebo or lifestyle modification in reducing body weight. They had a similar efficacy as the Western anti-obesity drugs but with fewer reported adverse effects."

[44] "The Bonghan system is a newly-discovered circulatory system, which corresponds to classical acupuncture meridians and was discovered in the early 1960s by Bonghan Kim."

[45] "Numerous experimental studies have demonstrated that acupuncture can correct various metabolic disorders such as hyperglycemia, overweight, hyperphagia, hyperlipidemia, inflammation, altered activity of the sympathetic nervous system and insulin signal defect, all of which contribute to the development of IR. In addition, acupuncture has the potential to improve insulin sensitivity."

There are many things in the biomedical literature of which one can be skeptical. The best way to handle it is to include everything, include every point of view, so that the relative strengths of the arguments can be sensed. Wnt (talk) 04:28, 20 February 2013 (UTC)

Are we talking about everything = primary studies here or everything = all secondary sources? I think we should go with the opposite approach, we should write the guideline to limit what we have to look at to only the smallest number of best-quality secondary sources. From those (much smaller in number than primary sources) we evaluate the sources and choose the best. Those we summarize in the article. My feeling is that this is the approach Wikipedia wants us to take everywhere. Zad68 04:33, 20 February 2013 (UTC)

How are we supposed to decide what are the best secondary sources until we've gone through them all? Why not let the best primary sources, from good journals, come in ahead of secondary sources we don't know the quality of? Where in this guideline does it tell you which "not quite the best secondary sources" are supposed to be deleted? Who has the right to evaluate which secondary sources are "best", and why don't they have the right to evaluate which primary sources are "best"? The way to do this isn't to play dowser and pick out the best sources, but to encourage people to just throw in a mountain of data and then organize it until a small summary comes out in the top-level article. Wnt (talk) 04:40, 20 February 2013 (UTC)

No, that clearly is wp:Original research. We don't choose based on our editors' thinking but rather on the type of sources. If recent, relevant, widely cited secondary sources of similar quality happen to disagree substantially, we report the different POVs in approximate proportion to the prevalence found. This is not specific to MEDRS, but applies to RS in general.LeadSongDog come howl! 05:23, 20 February 2013 (UTC)

All content should typically be based on secondary sources not just medical content. WP:MEDRS is already in line with WP:RS Doc James (talk · contribs · email) (if I write on your page reply on mine) 05:38, 20 February 2013 (UTC)

I don't see what isn't clear about WP:MEDRS applying to "medical and health-related content in any type of article"; specifying "incidence, prevalence, causes, and prognoses" and anything else that falls under medical/health therefore seems redundant to me. I also consider any health aspect/issue to be a medical aspect/issue, so stating "medical and health" also seems redundant to me...but I know that others often use "medical" in a stricter sense.

As for primary sources vs. secondary sources, like the guideline James pointed to states, primary sources are okay to use at times; that use should be sparingly, however, and articles should typically or largely be based on secondary sources. Obviously, WP:MEDRS is stricter on that than the general guideline. Flyer22 (talk) 07:02, 20 February 2013 (UTC)

But there are two things that I do think may need clarifying. The first is that WP:MEDRS also applies to anatomy not just when an anatomical part is malfunctioning and/or is diseased, which is why it is addressed in a general way at WP:MEDMOS. The second is whether or not WP:MEDRS applies to content that discusses historical medical practices, such as how society used to practice a medical procedure and/or what they thought of it and/or an anatomical part, how these things affected society; the consensus has been that WP:MEDRS generally does not apply in these cases; one example is the History and Society and culture sections of the Circumcision article. Flyer22 (talk) 07:26, 20 February 2013 (UTC)

My goal (for what it is worth) is to define the core matter more clearly - can we please argue about exceptions/refinements after we get the main thing nailed down? One thing at a time would be a more efficient way to go... trying to rush ahead and deal with side matter will make it impossible to get anything done. Can we please deal with that after we figure out what is the core matter constituting "biomedical information"/"health information" /"medical and health-related content"? Jytdog (talk) 14:05, 20 February 2013 (UTC)

I don't see anything confusing in the current scope. Has nobody got common sense? What is the problem with anatomy? I'm really struggling to see how anyone could have a problem sourcing this. Off the top of my head, the only anatomical things I can think of that might possibly appear in primary research papers these days are some of the more dubious findings from functional MRI claiming that, I don't know, love of poetry is found in the part of your brain next to your left ear. Do people really think "Oh no! I must find a literature review in a high quality journal that says "the knee bone connected to the thigh bone""? Are standard anatomy textbooks banned by MEDRS? Or historical practices? In ketogenic diet I discussed some historical thoughts on diet and epilepsy. Those come from high quality secondary sources as that is what all WP text is built upon. I, as footnotes, have cited the historical works as helpful bits of information for the reader, but in no way did I browse the historical archives of some dusty medical library and dig out this stuff from the parchments and translate it from the Greek, nor would I pretend to the reader that those were my "sources". Or controversial retracted papers? I can cite Wakefield's dubious work per WP:NOTRELIABLE perhaps to quote a bit or to get some dodgy numbers but I need reliable secondary sources to establish the weight to give to the topic for any given article, and to make any kind of comment about it. Can anyone give me an example of something they want to write on WP that MEDRS is currently making impossible because of some problem with the scope? -- Colin°^Talk 15:26, 20 February 2013 (UTC)

I see no chance to reach consensus on changing anything - too much passion and no room - no opening - to actually discuss anything. So I am outta here. Best regards,Jytdog (talk) 15:34, 20 February 2013 (UTC)

Colin, I mentioned above that "I don't see what isn't clear about WP:MEDRS applying to 'medical and health-related content in any type of article.'" However, I brought up the anatomy aspect because, like I also mentioned above, some people use the word "medical" in a stricter sense. For example: Most, maybe currently all, anatomical articles, like the Heart article, are not tagged with the WP:MED banner; they are instead tagged with the WP:ANATOMY banner because there are members of WP:MED who feel that the WP:ANATOMY tag is sufficient enough. And, indeed, sometimes when an anatomical topic that has nothing to do with malfunctioning or disease of the organ has been brought up at WP:MED, WP:MED has ignored the matter and/or directed the person to WP:ANATOMY. Sometimes, even things that have to do with sexual dysfunctions/sexual disorders are ignored by WP:MED, even though such things, including psychological/psychiatric aspects, do fall under "medical." And as has been mentioned at WP:ANATOMY before, WP:ANATOMY is not as active as WP:MED is (the same goes for WP:PSYCHOLOGY, WP:PSYCHIATRY and WP:SEX) and therefore listing the topic at WP:MED is more likely to be helpful; this is why it has been suggested that WP:ANATOMY be merged with WP:MED. As for the history/culture stuff I mentioned, that is due to some people not fully knowing when WP:MEDRS applies and when it does not; some people have acted as though WP:MEDRS applies to everything in an article about a medical topic; the Circumcision article is one example that people who believe that are wrong. Flyer22 (talk) 16:23, 20 February 2013 (UTC)

I am aware that the lead states: Sources for all other types of content—including all non-medical information in medicine-related articles—are covered by the general guideline on identifying reliable sources rather than this specific guideline.

But it sometimes appears that some people are not aware that WP:MEDRS makes that clear. Flyer22 (talk) 16:47, 20 February 2013 (UTC)

An example of a primary source important for anatomy is [46], in which the authors found that a tiny magnetic compass attached to the plasma membrane of 0.01%-0.04% of cells in the nose of a trout enable the fish to sense the magnetic field. So far as I know it hasn't been "reviewed" yet - unless you counted peer review, that is. As every cell of the body potentially could contain some rare organelle, and most of them would be biochemical things not as easy to recognize (!) as this, I would suggest that the vast majority of human anatomy may at this time still be undiscovered. Wnt (talk) 17:12, 20 February 2013 (UTC)

I'm curious, Wnt, if you believe that the primary research paper you found could be used on WP to support the statement "a tiny magnetic compass attached to the plasma membrane of 0.01%-0.04% of cells in the nose of a trout enable the fish to sense the magnetic field". Leaving aside that I can't find the "0.01%-0.04%" figure, the authors do not claim that this is how trout sense their magnetic field. They present (what they consider to be) "an effective method for isolating and characterizing potential magnetite-based magnetoreceptor cells" (emphasis mine). We need a reliable secondary source to determine if this, combined with all the other research, is sufficient evidence that they have in fact identified the cells trout use to sense a magnetic field. Such interpretation of basic research cannot be done by Wikipedians. This is simple policy and there's no need to even consult MEDRS to be aware of that. Colin°^Talk 17:41, 20 February 2013 (UTC)

It is true that I misspoke regarding this - I was not actually composing a sentence for an article, and would indeed have made more effort to stick to precisely what the source said for that. But when people discuss whether they should consider whether something "anatomy", or whether it is covered by MEDRS, they do not hold out for strictly sourced statements to prove that is true! (in other words, if the paper says that a drug "may be useful in treating obesity", you won't let me say "but the source doesn't say the drug does treat obesity, so it isn't a medical result!") Wnt (talk) 19:03, 20 February 2013 (UTC)

I'm not following your above sentence beginning "but when people discuss...". I do wish people would stop trying to find simple in/out yes/no answers to over-simplistic statements like "whether it is covered by MEDRS" or "it's anatomy" or "it's not anatomy" or "that's a primary source". These issues can't be resolved without clarifying what "it" is and what purpose the source is being used for, what the text is saying, what the surrounding article covers, etc, etc. Btw, most primary research papers contain boasts about what their findings "might be useful" for, and such boasts should be taken with a pinch of salt: they are written for funding bodies to read and open their wallets, and for the journal editors to consider whether to publish or not. Colin°^Talk 19:28, 20 February 2013 (UTC)

Yes major medical textbooks are reliable sources per WP:MEDRS thus pull out an anatomy text and write to your hearts content. Still no need to use primary research papers. Doc James (talk · contribs · email) (if I write on your page reply on mine) 18:29, 20 February 2013 (UTC)

Sorry for inadvertantly deleting a bunch of comments. I was responding to Ryanspir. I will try to be more careful. Desoto10 (talk) 04:09, 2 April 2013 (UTC)

requesting comments on article

Hi all. I was bold and revised the Bisphenol A article based on my understanding of the MEDRS guideline. The changes were reverted... I have not entered an edit war but instead am opening talk sections on each thing I deleted. Basic problem from my POV is a laundry list of toxicology issues under a heading called "Health Effects" - in violation of the letter and spirit of MEDRS. It is something I have struggled with for a while -- how to deal with tox issues; people very concerned about chemicals in the environment seem to love piling in laundry lists of "health issues" into articles about various chemicals (pesticides, etc) based on primary studies and this article is one of them. comments welcome!Jytdog (talk) 15:43, 6 March 2013 (UTC)

(On a related note), would referencing TOXNET/HSDB (seems to be a repository of publications) qualify as a non primary source? See [47] . Lesion (talk) 16:18, 6 March 2013 (UTC)

I would say no not of itself. It seems to be a rich search site, but there is no synthesis/evaluation in it per se. Looks like you can get to some good 2ndary sources through it.

Hops cures Cancer?

Hi, someone is trying to edit war to insert the claim that hops causes cancer. can some people comment at Talk:Breast_enlargement_supplements#sourced_text. IRWolfie- (talk) 10:16, 17 March 2013 (UTC)

Avoiding false impressions of a medical consensus where few secondary sources are available?

Is there any way consistent with WP:MEDRS as written to convey a lack of medical consensus on items cited in a secondary source if no secondary source that directly refutes the other secondary source can be found?Zebulin (talk) 21:07, 28 March 2013 (UTC)

If you're talking about Circumcision, you are beginning from a false premise, because a couple hundred review articles have been published on the subject of male circumcision in the last decade, not counting the textbooks and other secondary and tertiary sources available.

In general, though, if you've got three or more recent, high-quality secondary sources that address a given point, and 100% of these high-quality secondary sources say the same thing about the given point, then we assume that there actually is a medical consensus. So to give an example, if 100% of multiple, recent, high-quality secondary sources say that breast self-examination does not save lives in normal-risk women (and, as it happens, 100% of them do say this), then we assume that there is a medical consensus on this point, no matter what some editors believe and no matter what some fundraising organizations say. WhatamIdoing (talk) 22:34, 28 March 2013 (UTC)

if I was asking about circumcision specifically I'd discuss it there. I'd prefer to discuss WP:MEDRS in general here. How is this policy serving the mission of wikipedia? Would article accuracy not be better served by indicating that secondary sources support finding A and that there continues to exist multiple recent peer reviewed published primary sources which support finding not-A? Furthermore what of cases where some secondary sources believe a definitive conclusion can be drawn from available published evidence but where most do not. The former tends to get published more easily than the latter, and in many circumstances wikipedia would often indicate a medical consensus in such cases where in fact none exits. I am especially alarmed with presenting any secondary source publications that are not directly refuted by other secondary sources as simple scientifically accepted facts. Generally Wikipedia should only present findings as simple unqualified facts in this manner when any contrary findings are fringe. If there is demonstrable non-fringe published contention then the presentation of the information must be more nuanced to avoid overtly misleading the readers.Zebulin (talk) 02:01, 29 March 2013 (UTC)

How is this policy serving the mission of wikipedia?
By reducing the amount of misinformation in articles. For example, MEDRS makes it much easier to present the mainstream medical views as being the mainstream medical views, and MEDRS makes it much harder to misrepresent the views of activists either as being mainstream or as being equal to the mainstream view.
Would article accuracy not be better served by indicating that secondary sources support finding A and that there continues to exist multiple recent peer reviewed published primary sources which support finding not-A?
No. NPOV requires articles to defer to secondary sources. This was true before MEDRS was created, and would continue to be true even if MEDRS were deleted. (As a side note, if you can't find anti-circumcision secondary sources, you're not looking very hard.)
Furthermore what of cases where some secondary sources believe a definitive conclusion can be drawn from available published evidence but where most do not. The former tends to get published more easily than the latter, and in many circumstances wikipedia would often indicate a medical consensus in such cases where in fact none exits.
As far as Wikipedia is concerned, what's published is what exists. If the real world's publishing system is biased, then Wikipedia will strive to accurately and proportionately reflect that real-world bias. WhatamIdoing (talk) 02:32, 29 March 2013 (UTC)

(e/c) Zebulin, sorry, your initial question as stated is difficult to parse. What exactly are you getting at? Are you asking whether biomedical statements in articles should be made unattributed in Wikipedia's narrative voice if there is only one secondary source available to support it? Or if there are other secondary sources that corroborate but none can be found to contradict it? WAID's response is a good rule of thumb.

Your follow-up challenges WP:MEDRS and in particular WP:MEDREV, the guideline that says we must respect secondary sources and not hand-select individual primary sources that contradict the conclusions of secondary sources, as you appear to be proposing we do. The problem with this suggestion is that the quality of primary study data is extremely variable and perhaps a large majority of primary study evidence is poor, or its relevance to a particular topic is not obvious. We depend on secondary to do the selection, evaluation and synthesis of the primary sources for us. Leaving this to individual Wikipedia editors will result in articles turning into unnecessarily useless patchworks of contradictory statements. Using WAID's example, we'd end up with breast self-examination saying "this and this and this secondary source say breast self-exams do not save lives in normal-risk women, but this and this and this primary study say otherwise" and now we've made a useful article useless because those primary studies were flawed or didn't actually study normal-risk women. We need to use the secondary sources for their intended purpose, to draw conclusions from the primary data for us.

I'm not sure I'd agree with your assertion that secondary sources that provide conclusions necessarily get published more easily than ones that don't. By now I've looked at dozens and dozens and dozens of secondary sources, and lots of them are eager to point out that weak evidence from poorly-designed primary studies did not allow a conclusion to be drawn. Look at any large handful of Cochrane Library abstracts and see what I'm talking about.

If you'd like to pursue, can you please provide a specific example of a secondary source reviewing primary literature and drawing a conclusion, the primary sources that contradict the secondary, and the article content you're proposing. Zad68 02:55, 29 March 2013 (UTC)

Zebulin, you may have misunderstood what MEDRS is. MEDRS is not a special content guideline for medical claims; rather, MEDRS explains how WP:RS applies to medical claims. The difference, while perhaps subtle, is quite crucial. Jayjg ^(talk) 16:53, 29 March 2013 (UTC)

If secondary sources are critical of another secondary source in general terms but do not specifically refute the secondary source can we source the specific findings of the secondary source that is being criticized while in some way acknowledging the secondary source being used is itself under fire? As I stated in the section title I'm mainly concerned about MEDRS conveying a sense of scientific consensus where there is in fact anything but a consensus.Zebulin (talk) 17:52, 9 April 2013 (UTC)

Findings published in WP:MEDRS-compliant secondary sources like review articles or systematic reviews often have people commenting on them in the subsequent issues of the journals the reviews are found in, and they'll usually be labeled "Commentary". There can also be follow-up comments on the comments, comments on those comments, ad infinitum. These comments are generally not expected to be peer-reviewed sources, and so they'll probably be treated by WP:MEDASSESS as "expert opinion". As that places them at the bottom of our evidence hierarchy, we almost never include such commentary as sources in articles. If those doing the commenting get actual review articles or systematic reviews published in equally high-power journals, they can be used alongside the existing secondary sources cited. Zad68 18:12, 9 April 2013 (UTC)