Wikipedia talk:Identifying reliable sources (medicine)

From Wikipedia, the free encyclopedia
Jump to: navigation, search
Frequently asked questions (FAQ)

These are some Frequently Asked Questions about Wikipedia's guidelines on sourcing for medical content, manual of style for medicine-related articles, and how the guidelines and policies apply to biomedical content.

General
Why do you have special rules for medical information?
Different types of sources have different strengths and weaknesses. A type of source that is good for scientific information is not usually as reliable for political information, and vice versa. Since Wikipedia's readers may make medical decisions based on information found in our articles,[1] we want to use high-quality sources when writing about biomedical information. Many sources that are acceptable for other types of information under Wikipedia's general sourcing guideline, such as the popular press, are not suitable sources for reliable medical information.[2][3] (See also: WP:MEDPOP and WP:WHYMEDRS)
When do I need to follow MEDRS?
MEDRS-compliant sources are required for biomedical information. Like the policy on the biographies of living people ("BLP"), MEDRS applies to statements and not to articles: biomedical statements in non-medical articles need to comply with MEDRS, while non-medical statements in medical articles (such as in a History section) do not need to follow MEDRS. Also like BLP, the spirit of MEDRS is to err on the side of caution when making biomedical statements. Content about human biochemistry or about medical research in animals is also subject to MEDRS if it is relevant to human health.
Sourcing
I used a peer-reviewed source, but it was reverted, and the editor said I needed to use a review. I did, didn't I?
Probably not. Most peer-reviewed articles are not review articles. The very similar names are easily confused. For most (not all) purposes, the ideal source is a peer-reviewed review article.
Why can't I use primary sources?

Primary sources aren't completely banned, but they should only be used in rare situations. Whenever possible, you should cite a secondary source such as:

So if primary sources can be used in rare cases, what are those rare cases?

Primary sources might be useful in these common situations:

  • when writing about a rare disease, uncommon procedure, etc., for which no high-quality secondary literature is available, or for which the available secondary sources do not cover all of the information normally included in an encyclopedia article.
  • when mentioning a famous paper or clinical trial that made a recognized substantial impact, as part of a purely historical treatment of a topic.
  • when describing major research that has made a significant impact (i.e., continued and substantial coverage). While recent research results are normally omitted, it is sometimes necessary to include it for WP:DUE weight. In this case, it is usually preferable to read and cite the primary scientific literature in preference to WP:PRIMARYNEWS sources. Later, these primary sources can be replaced or supplemented with citations to high-quality secondary sources.
Can I use websites like Quackwatch?
Quackwatch is a self-published website by an author who is an expert in problems with complementary and alternative medicine. Whenever possible, you should use a scholarly source instead of Quackwatch. However, if no scholarly sources are available, and the subject is still notable, then it might be reasonable to cite Quackwatch with WP:INTEXT attribution to the POV.
Can I cite Chinese studies about Traditional Chinese Medicine?
As of 2014, there are concerns regarding positive bias in publications from China on Traditional Chinese Medicine.[4][5] Such sources should be used with caution. The problem also includes issues with the academic system in China.[6]
Can I cite NCCAM (now NCCIH)?
Yes, but again only with WP:DUE weight. Unlike other branches of the National Institutes of Health, which are generally accepted as authoritative in their fields, NCCAM has been the focus of significant criticism from within the scientific community.[7] Whenever possible, you should cite the established literature directly.
What if I can’t find any MEDRS-compliant sources on a subject?
MEDRS contains a section about finding sources which may be helpful. Alternatively, a more experienced editor may be able to help you find them (or to confirm that they do not exist).
Neutrality
What is a fringe medical claim?
How should fringe medical claims be described?
When fringe claims have been widely reported in the press, have a large popular following, and/or have a long history, it may be appropriate to describe them in terms of that reporting, popularity, or history. However, weight should be determined by MEDRS-compliant sources, and the context (or lack thereof) should not make implications about medical statements that are not supported by such sources. Guidance on the additional considerations relevant to fringe subjects can be found at WP:FRINGE, as well as at other places such as WP:WEIGHT and WP:EXCEPTIONAL.
In the case of alternative medicine, medical statements are often derived from an underlying belief system, which will include many propositions that are not subject to MEDRS. These propositions are subject to the usual sourcing requirements and the usual requirements for determining fringe status.
If a treatment hasn't been shown to work, can we say it doesn't work?

There are three possible situations:

  • no evidence exists (i.e., studies for the treatment have not been published, or the evidence base is too small or weak to draw any conclusions);
  • evidence exists, and it shows no effect;
  • evidence exists, and it shows an effect.

In the first case, we cannot say that it does not work, but we can say that there is no evidence to determine whether it works. In the second, we can say that there is no evidence that it works. In the last, we should say that there is evidence that it has an effect.

After multiple, high-quality independent studies have been published, it is reasonable to understand "no evidence that it works" as "some evidence that it does not work". You should follow the lead of review articles and other secondary sources for determining when this threshold has been crossed.

Reports may conflict with each other. For example, a clinical trial may produce no evidence of an effect, but the treatment's manufacturer might produce testimonials claiming a positive effect. You should follow the lead of review articles and other secondary sources for determining how to balance these claims.

Should medical content be attributed?
Why not say there is a call for more research?
How can Quackwatch be considered a reliable source?
Finding and using sources
How can I find good sources using PubMed?
Are there ways to find good sources other than PubMed?
Besides being a secondary source, what else indicates a source is of high quality?
I found what looks like a good source, but can't access the full text – what next?

Most scholarly journals are behind paywalls. Some options to access these articles include visiting a local university library, visiting The Wikipedia Library, and WikiProject Resource Requests.

Note that paywalled articles are frequently pirated and made available on the open web. When linking to a journal article, care must be taken not to link to such a pirate copy, as such a link would be a copyright violating link in contravention of Wikipedia's policy. In general if you find such a copy and it is not accompanied by text explicitly stating that it is made available with the permission of the copyright holder, assume that it is potentially infringing, and do not link to it. This holds for all edits in Wikipedia, not just in article space.

How do I reference a medical article?

Almost all medical articles medical articles are indexed by the PubMed search engine and have a Digital object identifier (DOI) assigned to them. All articles included in PubMed are assigned an eight-digit PubMed identifier (PMID). These identifiers can be used to refer to articles, which is preferred to URLs as it makes a reliable link which is resilient to changes beyond our control – i.e. the publisher being acquired by another publisher and it's "normal" web URLs changing as a consequence.

Once you have the PMID, there are a number of tools such as this one which you can use to generate a full citation automatically.

In article references, the "doi" and "pmid" parameters are preferred to the "url" parameter for such reasons.

On Talk pages, when referring to journal articles, is it good practice to make any link using these types of identifier also:

  • Typing "PMID", a space, and the 8-digit PubMed identifier will be automagically turned into a link by the Wikipedia software (e.g. PMID 21148220).
  • Any DOI can be turned into a resolvable web address by prepending "http://dx.doi.org/" to it (e.g. http://dx.doi.org/10.1136/bmj.c6801).
Conflict of interest
Are there special rules on conflicts of interest for health content?
What if I am being paid to edit medical content?
References

References

  1. ^ Laurent, MR; Vickers, TJ (2009). "Seeking health information online: does Wikipedia matter?". J Am Med Inform Assoc. 16 (4): 471–9. PMC 2705249Freely accessible. PMID 19390105. doi:10.1197/jamia.M3059. 
  2. ^ Schwitzer G (2008). "How do US journalists cover treatments, tests, products, and procedures? an evaluation of 500 stories". PLoS Med. 5 (5): e95. PMC 2689661Freely accessible. PMID 18507496. doi:10.1371/journal.pmed.0050095. Lay summaryGuardian (2008-06-21). 
  3. ^ Dentzer S (2009). "Communicating medical news—pitfalls of health care journalism". N Engl J Med. 360 (1): 1–3. PMID 19118299. doi:10.1056/NEJMp0805753. 
  4. ^ Li J, et al The quality of reports of randomized clinical trials on traditional Chinese medicine treatments: a systematic review of articles indexed in the China National Knowledge Infrastructure database from 2005 to 2012. BMC Complement Altern Med. 2014 Sep 26;14:362. PMID 25256890
  5. ^ Further information:
    • "Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." Vickers, Andrew (April 1, 1998), Do certain countries produce only positive results? A systematic review of controlled trials., Control Clin Trials, PMID 9551280 
    • Ernst, Edzard (2012). "Acupuncture: What Does the Most Reliable Evidence Tell Us? An Update". Journal of Pain and Symptom Management. 43 (2): e11–e13. ISSN 0885-3924. PMID 22248792. doi:10.1016/j.jpainsymman.2011.11.001. 
  6. ^ Qiu, Jane (January 12, 2010), Publish or perish in China, Nature 
  7. ^ Some examples:
    • Nature Reviews Cancer: "the subject of rancorous scientific and political debate over its mission and even continued existence"
    • Clinical Rheumatology: "The criticism repeatedly aimed at NCCAM seems justified, as far as their RCTs of chiropractic is concerned. It seems questionable whether such research is worthwhile."
    • Nature News: "still draws fire from traditional scientists", "Many US researchers still say such funding is a waste of time and money."
    • Science News: "[NCCAM] is a political creation"; "This kind of science isn't worth any time or money" (quoting Wallace Sampson)
    • Science Policy Forum: "[NCCAM] was created by pressure from a few advocates in Congress"; "NCCAM funds proposals of dubious merit; its research agenda is shaped more by politics than by science; and it is structured by its charter in a manner that precludes an independent review of its performance"; "NCCAM is unable to implement a research agenda that addresses legitimate scientific opportunities or health-care needs"
Other helpful resources

Graphics in #Books[edit]

For reference, here is the graphic (and associated caption) under discussion:


Books can often be a great source when starting an article, giving a good overview of the subject. However, all books are not equally good sources. This simplified flowchart shows which books make the cut for inclusion in Wikipedia articles.

I am a little concerned that the graphic in Wikipedia:Identifying_reliable_sources_(medicine)#Books seem to be misleading. It appears to conflate the "level" of a book with its quality or reliability. To be sure, it is unlikely that a low level textbook (e.g., the popular press book) is reliable under MEDRS, but it seems like a mistake to say that professional level books are higher quality than graduate level or college level books for example. The text of the guideline emphasizes the need for a book to be up to date, more than the specific level at which it is aimed. Professional books can be out of date, and current graduate level textbooks can be sources of the highest quality. So, for now I am removing the graphic. This material seems to be better conveyed by the written text, which the graphic fails to summarize. Sławomir Biały (talk) 10:51, 2 April 2017 (UTC)

In general the image holds true, and varying quality between different types of books is unrelated to whether they are up-to-date or not. This image doesn't touch upon whether it is better to use a graduate level textbook from 2005 over a undergraduate textbook from 2015. Do you have any suggestions for improvements in wording or in the image to clarify this? As far as I'm concerned the problem is in no way alleviated by removing a clear and informative image. Carl Fredrik talk 12:58, 2 April 2017 (UTC)
Really there are two issues with the image. One of which is the timeliness of the material, discussed in the text. The other is that it is quite simply a mistake to equate the intended level of a book with its factual reliability. I am sure that there are professional level books on chiropractic, for example, that would not be regarded as reliable medical sources. Sławomir Biały (talk) 13:00, 2 April 2017 (UTC)
Well, your second statement that you call a "mistake" generally holds true: that there is a correlation between intended level of a book and factual reliability/accuracy. Of course it does not always hold, but taken together with the section on academic publishers vs. predatory or popular publishers: you end up with a workable definition. I agree it might be inaccurate to equate this with quality, but it is a usable measure for suitability in most medical articles.
When it comes to pseudoscience, such books are seldom published by recognized publishers. Carl Fredrik talk 13:05, 2 April 2017 (UTC)
I'd like to add what I think is probably the most important point: the images are there to guide interest and maintain an ease of understanding. The text is more important — and to me if one more reader looks at the images and gets the jist of what we mean, rather than not reading this "dry" and "boring" guideline, then I'm happy. Better to have a superficial understanding of something than to have no understanding at all — that is what the images are for. The text will always trump the images.
Please come with suggestions and I can incorporate them quickly, the files are svgs and easily editable. Carl Fredrik talk 13:10, 2 April 2017 (UTC)
I am not convinced that college level and graduate level textbooks are generally less reliable than professional level books. Sławomir Biały (talk) 13:16, 2 April 2017 (UTC)
I am inclined to agree with Sławomir Biały here. The broad generalization that texts pitched at an undergraduate or graduate versus a "professional" audience are inherently or presumptively of lower quality is not really justified. At best, it might be a tolerable rule of thumb in an all-else-being-equal situation—but I've never yet encountered a situation where all else is really equal. On balance, the graphic is more likely to mislead than to assist. As with all sources, we should be examining textbooks and monographs on a case-by-case basis to determine if their use is appropriate (or sufficient) to support a given claim in a particular article. TenOfAllTrades(talk) 14:46, 2 April 2017 (UTC)
There is an incredible overestimation of the way general editors rely on guidelines here. Most (if not all) editors go by gut instinct, only consulting guidelines when there is an issue. Once again I will clarify that in general the heuristic holds, and only rarely is it not applicable. Thus it is a useful image to convey the most basic understanding of the guideline — that there exist different levels of published works, some of which are more suitable than others.
The other major issue here is that: removing the image does not solve any issues — the confusion is still present in the text. If you are suggesting we clarify that this is only a rule of thumb — fine, I agree that may be apt, but removing all visual aids indiscriminately is not helpful.
Please suggest some improvements instead, this image has been in the guideline for 8+ months without any critique. If we really need to remove it, it isn't something that will happen on a whim with a 1 paragraph justification on the talk-page. Carl Fredrik talk 16:14, 2 April 2017 (UTC)
I don't see how the length of time that the image was in the guideline is relevant. Was there some RfC that preceded its introduction? Otherwise one editor may justifiably remove such an image just as another may add it, without either editor being accused of behaving whimsically. Certainly, if I had been notified eight months ago, I would have voiced my concern at the time.
I am not aware of any specific number of paragraphs that must accompany the removal of a recently-added policy item, but I note an inability to find any paragraphs that specifically accompanied its addition. On the contrary, the only paragraph I can find in the archive that specifically addresses the new images argues against their inclusion:
It often seems a good idea to replace editorial judgement based on descriptive guidelines by a flowchart or other deterministic set of rules, but it very rarely turns out to be a good idea in practice. It leads to wikilawyering about the precise boundaries of the choices in the "rules". It discourages editors, especially new editors, from trying to understand the guidance provided, instead encouraging them to follow their interpretation of the "rules". — @Peter coxhead:
I note that these reasons are complementary to my own positive reasons, and also independently support the removal of the graphic. Let me note yet another reason the graphic is misleading: very often it is the case that textbooks at the graduate and undergraduate level are kept more up-to-date than books targeting a professional audience, because the former have a more frequent publication cycle because of the economics of the publishing industry, while the latter are much less likely to go into second and third editions and typically have a much longer cycle between editions. Sławomir Biały (talk) 17:16, 2 April 2017 (UTC)
3 quick points before I go to bed (will write a longer reply tomorrow):
  1. Stop it with the strawman about No. of paragraphs. The point is this is stable and you contest it — that isn't enough to just get rid of it.
  2. I disagree very strongly with the idea that images must be perfect to be included. The fact is no one knows all our policies. Not even the most experienced editor has up-to-date knowledge of all relevant policies. Images can guide both new and experienced users to better application of policies — even if they are not perfect. As long as there is an explanation that the image is not to be followed as set it stone, there really is no issue. Just point to the last sentence and then to the text, no Wikilawyering possible.
  3. Professional textbooks are actually quite often both published more often and are more up-to-date in medicine. This is different from other fields, but it is down to high demand on professional texts. Undergraduate or graduate textbooks often include simplifications or errors that are discredited in other sources: e.g. Precapillary sphincter — still taught in undergraduate courses today. @LT910001: Carl Fredrik talk 20:44, 2 April 2017 (UTC)
Claims of stable version are mostly spurious. Any version is "stable" until someone notices something wrong with it. And in that case, if it is wrong, then it is wrong, and better to make right rather than resort to an appeal to tradition (an informal fallacy, even for actually established traditions, let alone things recently added without consensus to an editing guideline). If someone had bothered to notify me eight months ago, I would have raised my objections then. Please consider an RfC in the future before adding images like these that do not actually summarize the consensus guideline. Sławomir Biały (talk) 10:41, 3 April 2017 (UTC)
Wikipedia does not operate as a series of RfCs. If you wish to keep up with the times you will need to monitor changes just like everyone else. There was absolutely nothing controversial about how this was inserted. Carl Fredrik talk 20:12, 3 April 2017 (UTC)
You added an image that, as far as I can tell, was not specifically discussed, and indeed was objected to at the time. The only item that you pointed to was that no one else has removed it in the relatively short time since you added it. Moreover, you added it in a sequence of edits, and did not clearly announce the change (which you should have anticipated would be controversial). Based on the current consensus, I believe that the image should be removed. You are, of course, free to establish the consensus that you claim existed by pointing to that consensus in a discussion archive, or by actually starting an RfC on the subject. Sławomir Biały (talk) 21:10, 3 April 2017 (UTC)
What objection are you referring to that predates your current one. Please suggest improvement, if we are simply debating whether to include an image or not we are stuck at an impasse. Carl Fredrik talk 22:18, 3 April 2017 (UTC) 
It is quoted in the comment above, and includes an attribution. One editor insisting that his own image must say in a policy page, against multiple editors saying it should be removed, is not an "impasse". Generally speaking, changes like this to policy require positive consensus. I do not see evidence that this was ever sought. Sławomir Biały (talk) 22:57, 3 April 2017 (UTC)
Two editors objecting on flawed grounds or that object only because something is not perfect hold less weight than the implicit consensus of the passage being included for 8 months. Neither is the image my own as it was created with the input and collaboration of a multitude of editors. There is no such praxis to require what you call "positive consensus" — such a phenomenon does not exist. Carl Fredrik talk 00:28, 4 April 2017 (UTC)
The "implicit consensus" appears to have been imagined. Surely, if there were such a consensus, then other editors in support of that consensus would have made their opinions known by now. Multiple editors have here objected to the graphic, and the only commentary I could find anywhere about this graphic in the discussion archive argued against its inclusion. You are, of course, still welcome to point to any discussion that took place regarding whether this graphic was an accurate summary of the guideline, in support of your contention that some consensus was reached. But it does seem to have been sneaked in amid a flurry of edits, without any explicit discussion on your part, and as a result the implications of the graphic appear to have been largely ignored until now. Sławomir Biały (talk) 09:30, 4 April 2017 (UTC)
I'd also point out that one can critique any image that tries to explain complex phenomena. The image on referencing guidelines is also not perfect. There exist guidelines published by professional organizations that trump national guidelines as well. E.g.: a guideline published by the APA (independent) is generally regarded as more reliable than one published by the Chinese government.
The images are not set in stone, but rather there to give a general idea of where a certain source falls on a continuum of sources. Maybe the best solution is to swap the arrows with some other form of denomination indicating a range of qualities. The same is true for crappy HTA reports, which exist and with the proliferation of HTAs are on the rise. That however does not mean that HTAs in general are not great sources, just that some aren't. Carl Fredrik talk 16:39, 2 April 2017 (UTC)
(ec)You argue that there is a "confusion...present in the text". I don't see it. The text certainly is less absolute in its suggestions. It offers very qualified, conditional guidance about which types of sources may be more robust under some circumstances, rather than sorting them into a strict pecking order—but that's a feature, not a bug. Your diagram implies a strong – if not absolutely rigid – ranking which does not capture the much more flexible and nuanced approach that we tend to follow when evaluating sources.
The improvement suggested, to be blunt, is to remove the misleading image. The absence of critique shouldn't be terribly surprising; you inserted it (as you have with other controversial and subsequently rejected changes) without making any comment on this talk page or engaging in any discussion. It was added as part of a flurry of more than 20 consecutive edits, large and small. Speaking for myself, I didn't even notice it had been added until today, when Sławomir Biały pointed it out and removed it. (Honestly, I expect we'll also have to have this same discussion about the "Referencing a Guideline" figure which you added in the same flurry, without so much as an edit summary: [1].) TenOfAllTrades(talk) 17:00, 2 April 2017 (UTC)
I'm sorry, what do you mean by other controversial changes that were reverted? That is not something I am aware of, and I don't think any of those who frequent this page ave seen any such edits. I don't have time right now to explain why most of the arguments against the image here are false — but they are and I can give you the full picture soon (especially a strong refutation of the previous remarks by Slawomir immidiately above this thread). Also be aware there is considerable discussion on WT:MED, so that once discussion occurs here it is often fleeting and more of a rubber stamp. If you wish to keep up you may want to follow that talk-page, and be aware of what is tacit consent (stability over a year is more than enough to imply that). Guidelines are not static, and if you do not wish to reread them from time to time you may want to keep up with their evolution in real time. The solutions is not to go back to prior versions just because you missed the changes. Carl Fredrik talk 20:32, 2 April 2017 (UTC)
The most significant example(s) would be from September (and then October) of last year, when you added a table that tried to summarize criteria for evaluating sources. The first time around, you added the table without any discussion; another editor (User:Jytdog) moved it to the talk page to discuss: Wikipedia talk:Identifying reliable sources (medicine)/Archive 26#Proposed MEDDUE section. A month later, you posted a modified version to the talk page to seek consensus (a good idea) but then you went ahead and added your preferred version to the article over unanimous objection on the talk page (bad) and then edit warred to try to keep it in the guideline (worse), only stopping when I told you the next step would be a review of your conduct at AN/I. TenOfAllTrades(talk) 15:22, 3 April 2017 (UTC)
That is an extremely inaccurate summary of what happened, and entirely ignores most of the picture. Carl Fredrik talk 20:14, 3 April 2017 (UTC)

I think some other editors need to weigh in, but so far I count two opposed to inclusion of the image, and one supporting it (who was also the same editor who added the image in the first place). Additionally, the above quotation from the archive appears to show that the image was not added with consensus to begin with. Ordinarily, this would count as a clear consensus against the image. Was there any place in the discussions that took place where some editor besides Carl Fredrik commented positively on this image? Sławomir Biały (talk) 10:35, 3 April 2017 (UTC)

If the image is intended to summarise the policy it should also include timeliness. Readability of the source in my view should be less of a concern than its authoritativeness. There is always the option of adding a "lay summary" to the reference. JFW | T@lk 12:38, 3 April 2017 (UTC)
Would a suitable correction be to change quality -> authority (in the image) and to amend the section on timeliness/weighing sources, Jfdwolff?
We could mention that each source falls on a continuum of different factors that make it good/bad. Incidentally TenOfAllTrades, this is exactly what was proposed in October that went through a period of contention before being place on hold (mainly because I did not have time to improve it, not because of any threats). We could still pick this up and work on the following:
Proposed MEDUE section

Any source used in a medical article is weighed on multiple axes, with: quality of the source; authority of the source; date of publishing; and relevance; all being important. Each must be taken into consideration.

  Relevance   Quality   Age   Best possible
Appropriate Same topic AND High AND <2 years OR Yes
High
Related topic OR Medium-high OR 2-5 years OR One of several
Same field OR Medium-low OR 6-9 years OR One of many
Inappropriate Unrelated OR Low OR 10+ years OR No
Term  Unrelated   Low quality   Old   Better source
available
Amble belonging to the previous discussion that needs some rewriting to take into account a number of points raised then and now
  • A relevant source is one where the topic is the same as for the statement you use it for. E.g. — if you're writing about methotrexate treatment for rheumatoid arthritis: the best possible source in one covering treatment options in general, or even specifically methotrexate. The second best is a source that is about a different treatment, but states your topic in passing: "DMARDs have benefits over methotrexate, which may cause leukopenia". Less relevant sources may be a paper on psoratic arthritis that mentions methotrexate as a treatment for RA. Entirely unrelated or marginally related sources should never be used.
  • Quality assessment is performed according to the section on assessing evidence quality, but in essence means that higher quality sources trump lower quality sources — where the lowest quality sources should be entirely avoided.
  • Medical research moves in fits and starts: sometimes the state of published evidence can change dramatically from one day to the next, at other times progress is excruciatingly slow. When comparing review articles and if all else is the same — more recent articles should be chosen. Major organizations often recommend reviews of the evidence be no older than 2 years in order to be deemed "best evidence" (providing they haven't been trumped by newer authoritative evidence). rare diseases or areas where research moves slower are exceptions where Wikipedia takes a more pragmatic approach. However, sources older than 10 years should not be used (not including certain related fields such as anatomy, where the body of knowledge moves somewhat slower).
  • Always seek the best source possible. It is unreasonable to expect editors to always use the best sources, but they should always try. Issues finding or using the best sources may be: pay-walls (see the Resource Exchange for help on accessing pay-walled content); difficulties in determining which source is the best; or it may just be unreasonably time-consuming to go through all possible sources.
    When you have several contenders for best possible source, and they don't agree on something make sure to give both viewpoints, but remember to avoid giving false balance.

Try to avoid bias when selecting sources. Bias can arise from your search criteria and search tools, and in which sources you have access to (see for example FUTON bias).

Best, Carl Fredrik talk 20:20, 3 April 2017 (UTC)

One alternative update:
  Relevance   Authority *   Quality *   Age   Best possible   Suitability
Appropriate Same topic AND High AND High AND < 2 years
Yes
High
Related topic OR Medium-high OR Medium-high OR 2–5 years OR One of several
Medium
Same field OR Medium-low OR Medium-low OR 6–10 years OR One of many
Low
Inappropriate Different field OR Low OR Low OR Significantly older OR No
Unsuitable
Reason for
rejection
Unrelated   Low authority   Low quality   Old   Better source
available
  Unsuitable
  For each source consider the lowest possible score when judging suitability. This table is not exhaustive and other factors may come into play. Whether a source is the best possible or among the best possible is the most important factor.
  *See text on how to judge authority and quality
Needs some reformatting to make it look decent
Best, Carl Fredrik talk 20:35, 3 April 2017 (UTC)
  • This is not bad. Not sure where to comment on the book graphic discussion with this now brought in. I am ambivalent about the graphic but I feel very strongly that WP:MEDRS needs to remain very stable as we would be completely fucked without it; i can't imagine the struggles that people went through while MEDRS was being built and consensus created for it. Its strength flows from the widespread consensus it holds. It is not a rulebook to be imposed -- it expresses consensus, broadly held. Do not destabilize it. Do not try to force changes into it. Jytdog (talk) 00:42, 4 April 2017 (UTC)
Then we should not be deleting stuff that has been in there for nearly a year, just because one user opposes it. The introduction seems non-controversial, and it was in there for 8 months — if we want stability we should restore the image until consensus is achieved whether to include it or not. Anything else is in fact destabilizing. Carl Fredrik talk 00:47, 4 April 2017 (UTC)
Am not going to rehash issues around your behavior at this guideline from last year. Jytdog (talk) 00:52, 4 April 2017 (UTC)
Yeah, there are very different ways to look at that. Reverting based solely on the argument of no-consensus is disruptive — which is what you were doing last year. The issue is just as much about your behavior, and I don't think it is helpful to drag that into the picture, especially considering what other issues exist.
That I hold this guideline dear and find it important to keep it current so that we aren't stuck in a status quo that permits horrible misreadings (such as the no peer-review section, with unfortunately builds upon a fundamental misunderstanding of peer-review is — hence the horrible nature of that section and the Wikilawyering it allows: because it allows us to debate what peer review is) or let poor wording hang around: does not count against anything.
Also if anyone here were to look at the history, most of the edits on this guideline are reverts in some form, that is simply due to the nature of it — and hardly anything controversial: when most of what is being changed is done by those who have no idea what they are doing, or are trying to sneak in loopholes permitting them to reference their favorite quack-journal. What however is unusual is to remove long-standing content — thus it requires discussion, and hopefully in such a manner that it should improve the guideline in the end, not restore it to some status quo of 12 months ago. Carl Fredrik talk 01:03, 4 April 2017 (UTC)
Carl, while considering these changes to guide book source quality, can we have a look at the pyramids in WP:MEDASSESS, perhaps with the idea of consolidating the table information above into a graphical pyramid? For MEDASSESS itself, I've thought for some time that those references are old and require the user to interpret between them. Perhaps they could be synthesized into one graphic of our making. --Zefr (talk) 01:41, 4 April 2017 (UTC)
As an administrative note, this discussion should probably be carved into a new thread, since it seems to be only tangentially related to the now-removed image at the top of the thread. It proposes making a substantial addition to the body of MEDRS, and deserves to be presented in a way that ensures maximum exposure (and avoids the problem CFCF/Carl encountered above, where a change was introduced without discussion, and then undone when noticed months afterward.)
That said, the proposed table is very similar to one which CFCF introduced in September/October last year, and which was roundly rejected in a discussion at the time. See Wikipedia talk:Identifying reliable sources (medicine)/Archive 26#Proposed MEDDUE section. Briefly, I see three issues.
  1. Much of the table is just padded with filler: extra words to fill in boxes, without helping an editor to understand or apply MEDRS. For example, it's reasonable (though not terribly helpful) to say that we prefer high-quality sources. It's pointless padding to create a four-point, color-coded scale, with scoring categories "high", "medium-high", "medium-low", and low", and no specific guidance beyond a pointer (in the explanatory notes below) to MEDASSESS. The "Authority" column has the same problem.
    The "Relevance" and "Best possible" columns do little more than restate straight-up common sense. This isn't specific to MEDRS; for any claim on Wikipedia we strive to use the best possible sources which are directly relevant to the topic.
  2. The non-'filler' content may be misleading or actively harmful. Introducing the concept of a "score" has a long and troubled history, where inexperienced editors who don't have the background to read the medical literature knowledgeably and competently instead check by rote to see if a source 'ticks all the boxes'. The issues surrounding the 5-year rule of thumb and the problems with setting out very specific age strata have been extensively criticized, including the last time around.
  3. If we drop the filler and the misleading stuff, the entire table is readily replaced by one sentence along the lines of "In choosing sources, we prefer high-quality, reputable, relevant, recent publications."
In other words, it's drawing a picture for drawing a picture's sake, while adding in some new and unhelpful concepts not actually intended to be part of MEDRS. Lest CFCF continue to mistaken silence for agreement, I will explicitly note here that I strongly disagree with the addition of this or any substantially similar table or graphic. TenOfAllTrades(talk) 14:59, 6 April 2017 (UTC)
Just about everything you just said is wrong:
  1. This is highly relevant to the above discussion and has to do with the same issues that the image is meant to address. The jury is also still out on whether to include the image or make some minor changes to it. Please do not assume that things here are static or that we have now reached a consensus, we really have not.
  2. No, we could not replace it with a single sentence, however, we could replace it with a paragraph. But the problem there is that people don't read paragraphs of texts wedged in the middle of policy documents. MEDRS needs updating and your attitude is not helpful, and I would suggest that those who do not have at least some experience of editing medical articles not come with comments on what could be "dangerous" or not. Regarding the 5 year rule, that has been both criticized and lauded, with some of this guideline being far stronger in suggesting adherence to a 5 year cutoff. Neither does the current iteration even mention such a cutoff — it simply summarizes what consensus exists in high-level scientific literature that evidence changes over time. There are a multitude of sources for this, and I am growing ever more frustrated that editors with no experience in the field believe they can convincingly argue that there is no reason for such a cutoff. Don't make such statements without sources, and if we really want to debate it we need to go to the sources (it is a far to large discussion to be banded together with the rest of this — which is why no cutoff is included in the current iteration!)
  3. For the record, I strongly contest the notion that anything at all in that table is misleading. Whether some of it is a reiteration of common sense or not is beside the point. What is common sense to some is not common sense to others, and we are constantly stuck with problems where quacks try to subvert common sense with nonsense — hence: a good, easy to understand image is very, very helpful when directing new or bad faith editors to policies. Whether or not the words "medium high/medium low" need to be included is a very minor issue — the reason they are included is to indicate that a source falls along a continuum, and there are not only the high/low authority, reliability etc. levels. If there are other ways to indicate this — nothing is stopping us from using these instead — and I will reiterate: come with suggestions instead, not just "remove everything, because it isn't perfect".
Policies aren't written for experienced Wikipedia-editors, they are written to guide those that do not have the experience to intuitively know when and where a certain rule can or should be applied. Images are not superfluous, and do not exist for their own sake — they exist because they help readers remember and get a quick overview of the contents of the text. Do not assume that everyone reads every sentence of text. The specific paragraphs of text are important when referencing them in disputes, but far more important in general editing is the jist of a guideline: the way ordinary editors understand it. And here images are extremely helpful. So unless something is actively wrong (I do not think it is, and most if not all criticism here is debunked) — there is very good reason to include images.
There is no mistake in seeing the stability of a section over years as tacit consent.
What is a mistake is to see minor fluctuations in which non-regular editors attend to this guideline as implication of the state of consensus over time. Carl Fredrik talk 10:33, 7 April 2017 (UTC)
I'm not sure what qualifies as a "non-regular editor". We've both been editing Wikipedia for about as long. Is the contention that an editor who isn't carefully watching the day-to-day changes of a guideline is not a "regular editor" of Wikipedia, and therefore their opinion counts for less than a "regular editor" who (apparently) has a license to add things to guidelines without consensus? In any case, presumably the "regular editors" who supposedly form this consensus that you have repeatedly claimed have had a more than ample chance to comment here. Their silence is deafening. Sławomir Biały (talk) 12:21, 7 April 2017 (UTC)
  I believe it was clear that I was referring to regulars to this guideline page/talk page. 16:14, 7 April 2017 (UTC)
Stated for simple analogy, MEDRS is a traffic jam for less experienced editors to navigate, and more experienced editors rely on it being clear and intuitive enough to substantiate chosen sources or reject poorly sourced statements. Tables and graphics can be to MEDRS as highway and intersection signs are to traffic. There is abundant text for more detailed guidance. I support the clarity and immediate visual interpretation that color graphics can offer. --Zefr (talk) 15:11, 7 April 2017 (UTC)
An excellent analogy, that captures my feelings towards the guideline entirely. Carl Fredrik talk 16:14, 7 April 2017 (UTC)

Mismatch[edit]

"Supplemental Vitamin E and selenium can increase the risk of prostate cancer." (citing PMID 23552052)

This example sentence does not match the results/discussion of selenium in the cited source (per conclusions here).

Given the nuanced discussion of the (inconclusive) findings for selenium, perhaps better to restrict the example to Vitamin E. 86.186.155.183 (talk) 08:37, 13 June 2017 (UTC)

Another (basic) conceptual mismatch fixed here (see PMID 20924966). 86.186.155.183 (talk) 10:07, 13 June 2017 (UTC)
Thanks. fixed further. Jytdog (talk) 15:28, 13 June 2017 (UTC)

Edit request[edit]

Maybe someone qualified would be able to fix the paragraph about health technology assessments that follows the section on medical and scientific organizations. You could add an asterisk after some British organization who claims the orphan, or omit the paragraph. Thank you. -SusanLesch (talk) 19:17, 3 July 2017 (UTC)

Not done That page is not protected and does not require an administrator to update. — xaosflux Talk 20:16, 3 July 2017 (UTC)
Hi, Xaosflux. Correct. However we need a qualified editor with knowledge of the UK health system to make the edit. Pardon me for choosing a template that Wikipedia has. -SusanLesch (talk) 23:04, 3 July 2017 (UTC)
I have to admit I do not understand your request either. Carl Fredrik talk 23:35, 3 July 2017 (UTC)
First, sorry, @Xaosflux: I imagine that template set off an alarm somewhere that you answered! Second, CFCF. What don't you understand? The guidelines have an orphan asterisked section referring to nothing that needs attention. -SusanLesch (talk) 23:41, 3 July 2017 (UTC)
See the image. It has nothing to do with the UK. Carl Fredrik talk 00:09, 4 July 2017 (UTC)
Thank you, @CFCF: I missed that HTA! I leave it to you guys to decide if it makes more sense as part of the caption. (I don't think we have HTAs in the U.S. but maybe we do!) -SusanLesch (talk) 00:14, 4 July 2017 (UTC)
@SusanLesch: yes, we have a few of those - ensuring that protection does not get in the way of edits that improve the project is an import task for admins. — xaosflux Talk 00:22, 4 July 2017 (UTC)
@Xaosflux: thank you for being an emergency responder. I will try not to send a false alarm ever again Very sorry.svg Also I learned that the World Health Organization thinks the United States does have HTAs. -SusanLesch (talk) 14:05, 4 July 2017 (UTC)
@SusanLesch: never worry about that - I'd rather there be a bunch of false positives then block helpful edits! — xaosflux Talk 14:09, 4 July 2017 (UTC)