Evidence-based medicine (EBM) is defined as "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients." Trisha Greenhalgh and Anna Donald define it more specifically as "the use of mathematical estimates of the risk of benefit and harm, derived from high-quality research on population samples, to inform clinical decision-making in the diagnosis, investigation or management of individual patients." To broaden its application from individual patients to health care services in general and to allied health professions, it is also known as Evidence Informed Healthcare or Evidence Based Health Care or Evidence Based Practice.
In practice, clinicians contextualize the best available research evidence by integrating it with their individual clinical expertise and their patient's values and expectations. The incorporation of patient values and clinical expertise in EBM partly recognizes the fact that many aspects of health care depend on individual factors. These include variations in individual physiology and pathology, and quality-of-life and value-of-life judgments. These factors are only partially subjected to scientific inquiry and sometimes even cannot be assessed in controlled experimental settings. Application of available evidence is therefore dependent on patient circumstances and preferences, and remains subject to input from personal, political, philosophical, religious, ethical, economic, and aesthetic values. This has led to a shift from the original term Evidence 'Based' Medicine to Evidence 'Informed' Healthcare, to emphasize that decisions need not necessarily be based or complying with the evidence.
The broad field of Evidence Based Medicine would include rigorous and systematic analysis of published literature to synthesize high quality evidence, such as systematic reviews. It could also be referring to a medical ‘movement’, where advocates work to popularize the method and usefulness of the practice of EBM in the public, patient communities, educational institutions, and continuing education of practicing professionals.
- 1 Background and definition
- 2 Process and progress
- 3 Assessing the quality of evidence
- 4 Statistical measures
- 5 Quality of clinical trials
- 6 Limitations and criticism
- 7 Assessing the teaching of evidence-based medicine
- 8 In psychiatry
- 9 History
- 10 EBM and ethics of experimental or risky treatments
- 11 Application of the evidence based model on other public policy matters
- 12 See also
- 13 References
- 14 Bibliography
- 15 External links
Background and definition
Evidence-based medicine (EBM) has evolved from the need to bridge the gap between research and practice. EBM applies research information (evidence) to clinical practice, emphasizing the importance of the use of quantitative (as well as qualitative) evidence in the "art" of clinical decision making. It aims to make decision making more structured and objective by better reflecting the evidence from research. By introducing the use of research information in clinical decision making, particularly from clinical epidemiology, EBM has driven a transformation of clinical practice and medical education.
In 1996 David Sackett wrote that "evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients." This definition, put forward by one of the original proponents of evidence-based medicine, has since been adopted by major organizations, including the Cochrane Collaboration and the Centre for Evidence Based Medicine.
Evidence-based health service
An evidence-based health service tends to generate an increase in the competence of health service decision makers and is the practice of evidence-based medicine at the organizational or institutional level. It strengthens the motivation of any health service decision-maker to use scientific methods when making a decision and details of this approach to health services and public health has been discussed in a book titled evidence-based healthcare & public health. Research by Michael Fischer and colleagues at the University of Oxford finds that evidence-based rules may not readily 'hybridise' with experience-based practices orientated towards ethical clinical judgement, and can lead to contradictions, contest, and unintended crises. In their recent, large study of the UK health knowledge economy, they find the most effective 'knowledge leaders' (managers and clinical leaders) use a broad range of management knowledge in their decision making, rather than just formal evidence. Evidence-based guidelines may provide the basis for 'governmentality' in health care and consequently play a central role in the distant governance of contemporary health care systems.
Evidence-based decision making
The results of population-based research form the foundation of evidence-based medicine. It aims to use the experience of a population of patients reported in the research literature to guide decision making in practice. This practice of evidence-based medicine requires the application of population-based data to the care of an individual patient. In the past, we have relied on the experience of physicians or other health care workers to make decisions about therapy. In the current information era, this approach would be suboptimal as health care workers rapidly find themselves unable to cope with the influx of a huge variety of new information, from the irrelevant to the very important. Therefore, evidence-based decision making gradually emerged as a solution to integrate the best research evidence with clinical expertise and patient values and expectations as practiced by the individual health care provider. The concepts and ideas attributed to and labeled collectively as EBM/EBHC have now become a part of daily clinical lives, and health care professionals increasingly hear about evidence-based guidelines, care paths, questions and solutions. The controversy has shifted from whether to implement the new concepts to how to do so sensibly and efficiently, while avoiding potential problems associated with a number of misconceptions about what EBM/EBHC is and what it is not. The EBM/EBHC-related concepts of hierarchy of evidence, meta-analyses, confidence intervals, study design, and so forth are so widespread, that health care professionals have no choice but to become familiar with EBM/EBHC principles and methodologies.
Process and progress
The five steps of EBM in practice were first described in 1992 and the experience of delegates attending the 2003 Conference of Evidence-Based Health Care Teachers and Developers was summarized into five steps and published in 2005. This five step process can broadly be categorized as:
- Translation of uncertainty to an answerable question and includes critical questioning, study design and levels of evidence
- Systematic retrieval of best evidence available
- Critical appraisal of evidence for internal validity that can be broken down into aspects regarding:
- Systematic errors as a result of selection bias, information bias and confounding
- Quantitative aspects of diagnosis and treatment
- The effect size and aspects regarding its precision
- Clinical importance of results
- External validity or generalizability
- Application of results in practice
- Evaluation of performance
Using techniques from science, engineering and statistics, such as the systematic review of medical literature, meta-analysis, risk-benefit analysis, and randomized controlled trials (RCTs), EBM aims for the ideal that healthcare professionals should make "conscientious, explicit, and judicious use of current best evidence" in their everyday practice. Ex cathedra statements by the "medical expert" are considered to be the least valid form of evidence. All "experts" are now expected to reference their pronouncements to scientific studies.
The systematic review of published research studies is a major method used for evaluating particular treatments. The Cochrane Collaboration is one of the best-known, respected examples of systematic reviews. Like other collections of systematic reviews, it requires authors to provide a detailed and repeatable plan of their literature search and evaluations of the evidence. Once all the best evidence is assessed, treatment is categorized as (1) likely to be beneficial, (2) likely to be harmful, or (3) evidence did not support either benefit or harm.
A 2007 analysis of 1016 systematic reviews from all 50 Cochrane Collaboration Review Groups found that 44% of the reviews concluded that the intervention was likely to be beneficial, 7% concluded that the intervention was likely to be harmful, and 49% concluded that evidence did not support either benefit or harm. 96% recommended further research. A 2001 review of 160 Cochrane systematic reviews (excluding complementary treatments) in the 1998 database revealed that, according to two readers, 41.3% concluded positive or possibly positive effect, 20% concluded evidence of no effect, 8.1% concluded net harmful effects, and 21.3% of the reviews concluded insufficient evidence. A review of 145 alternative medicine Cochrane reviews using the 2004 database revealed that 38.4% concluded positive effect or possibly positive (12.4%) effect, 4.8% concluded no effect, 0.69% concluded harmful effect, and 56.6% concluded insufficient evidence.
Assessing the quality of evidence
Evidence quality can be assessed based on the source type (from meta-analyses and systematic reviews of triple-blind randomized clinical trials with concealment of allocation and no attrition at the top end, down to conventional wisdom at the bottom), as well as other factors including statistical validity, clinical relevance, currency, and peer-review acceptance. Evidence-based medicine categorizes different types of clinical evidence and rates or grades them according to the strength of their freedom from the various biases that beset medical research. For example, the strongest evidence for therapeutic interventions is provided by systematic review of randomized, triple-blind, placebo-controlled trials with allocation concealment and complete follow-up involving a homogeneous patient population and medical condition. In contrast, patient testimonials, case reports, and even expert opinion (however some critics have argued that expert opinion "does not belong in the rankings of the quality of empirical evidence because it does not represent a form of empirical evidence" and continue that "expert opinion would seem to be a separate, complex type of knowledge that would not fit into hierarchies otherwise limited to empirical evidence alone.") have little value as proof because of the placebo effect, the biases inherent in observation and reporting of cases, difficulties in ascertaining who is an expert and more.
U.S. Preventive Services Task Force (USPSTF)
Systems to stratify evidence by quality have been developed, such as this one by the United States Preventive Services Task Force for ranking evidence about the effectiveness of treatments or screening:
- Level I: Evidence obtained from at least one properly designed randomized controlled trial.
- Level II-1: Evidence obtained from well-designed controlled trials without randomization.
- Level II-2: Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group.
- Level II-3: Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence.
- Level III: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.
UK National Health Service
The UK National Health Service uses a similar system with categories labeled A, B, C, and D. The above Levels are only appropriate for treatment or interventions; different types of research are required for assessing diagnostic accuracy or natural history and prognosis, and hence different "levels" are required. For example, the Oxford Centre for Evidence-based Medicine suggests levels of evidence (LOE) according to the study designs and critical appraisal of prevention, diagnosis, prognosis, therapy, and harm studies:
- Level A: Consistent Randomised Controlled Clinical Trial, cohort study, all or none (see note below), clinical decision rule validated in different populations.
- Level B: Consistent Retrospective Cohort, Exploratory Cohort, Ecological Study, Outcomes Research, case-control study; or extrapolations from level A studies.
- Level C: Case-series study or extrapolations from level B studies.
- Level D: Expert opinion without explicit critical appraisal, or based on physiology, bench research or first principles.
Categories of recommendations
In guidelines and other publications, recommendation for a clinical service is classified by the balance of risk versus benefit of the service and the level of evidence on which this information is based. The U.S. Preventive Services Task Force uses:
- Level A: Good scientific evidence suggests that the benefits of the clinical service substantially outweigh the potential risks. Clinicians should discuss the service with eligible patients.
- Level B: At least fair scientific evidence suggests that the benefits of the clinical service outweighs the potential risks. Clinicians should discuss the service with eligible patients.
- Level C: At least fair scientific evidence suggests that there are benefits provided by the clinical service, but the balance between benefits and risks are too close for making general recommendations. Clinicians need not offer it unless there are individual considerations.
- Level D: At least fair scientific evidence suggests that the risks of the clinical service outweighs potential benefits. Clinicians should not routinely offer the service to asymptomatic patients.
- Level I: Scientific evidence is lacking, of poor quality, or conflicting, such that the risk versus benefit balance cannot be assessed. Clinicians should help patients understand the uncertainty surrounding the clinical service.
GRADE working group
A newer system was developed by the GRADE working group and takes into account more dimensions than just the quality of medical research. It requires users of GRADE (short for Grading of Recommendations Assessment, Development and Evaluation) who are performing an assessment of the quality of evidence, usually as part of a systematic review, to consider the impact of different factors on their confidence in the results. Authors of GRADE tables, grade the quality of evidence into four levels, on the basis of their confidence in the observed effect(a numerical value) being close to what the true effect is. The confidence value is based on judgements assigned in five different domains in a structured manner. The GRADE working group defines 'quality of evidence' and 'strength of recommendations' based on the quality as two different concepts which are commonly confused with each other.
Systematic reviews may include Randomized Controlled trials that have low risk of bias, or, observational studies that have high risk of bias. In the case of Randomized controlled trials, the quality of evidence is high, but can be downgraded in five different domains.
- Risk of bias: Is a judgement made on the basis of the chance that bias in included studies have influenced the estimate of effect.
- Imprecision: Is a judgement made on the basis of the chance that the observed estimate of effect could change completely.
- Indirectness: Is a judgement made on the basis of the differences in characteristics of how the study was conducted and how the results are actually going to be applied.
- Inconsistency: Is a judgement made on the basis of the variability of results across the included studies.
- Publication bias: Is a judgement made on the basis of the question whether all the research evidence has been taken to account.
In the case of observational studies, the quality of evidence starts of lower and may be upgraded in three domains in addition to being subject to downgrading.
- Large effect: This is when methodologically strong studies show that the observed effect is so large that the probability of it changing completely is less likely.
- Plausible confounding would change the effect: This is when despite the presence of a possible confounding factor which is expected to reduce the observed effect, the effect estimate still shows significant effect.
- Dose response gradient: This is when the intervention used becomes more effective with increasing dose. This suggests that a further increase will likely bring about more effect.
Meaning of the levels of Quality of evidence as per GRADE
- High Quality Evidence: The authors are very confident that the estimate that is presented lies very close to the true value. One could interpret it as: there is very low probability of further research completely changing the presented conclusions.
- Moderate Quality Evidence: The authors are confident that the presented estimate lies close to the true value, but it is also possible that it may be substantially different. One could also interpret it as: further research may completely change the conclusions.
- Low Quality Evidence: The authors are not confident in the effect estimate and the true value may be substantially different. One could interpret it as: further research is likely to change the presented conclusions completely.
- Very low quality Evidence: The authors do not have any confidence in the estimate and it is likely that the true value is substantially different from it. One could interpret it as: New research will most probably change the presented conclusions completely.
Guideline panelists may make Strong or Weak recommendations on the basis of further criteria. Some of the important criteria are:
- Balance between desirable and undesirable effects (not considering cost)
- Quality of the evidence
- Values and preferences
- Costs (resource utilization)
Despite the differences between systems, the purposes are the same: to guide users of clinical research information on which studies are likely to be most valid. However, the individual studies still require careful critical appraisal.
Evidence-based medicine attempts to express clinical benefits of tests and treatments using mathematical methods. Tools used by practitioners of evidence-based medicine include:
The pre-test odds of a particular diagnosis, multiplied by the likelihood ratio, determines the post-test odds. (Odds can be calculated from, and then converted to, the [more familiar] probability.) This reflects Bayes' theorem. The differences in likelihood ratio between clinical tests can be used to prioritize clinical tests according to their usefulness in a given clinical situation.
The area under the receiver operating characteristic curve (AUC-ROC) reflects the relationship between sensitivity and specificity for a given test. High-quality tests will have an AUC-ROC approaching 1, and high-quality publications about clinical tests will provide information about the AUC-ROC. Cutoff values for positive and negative tests can influence specificity and sensitivity, but they do not affect AUC-ROC.
Number needed to treat / harm
Number needed to treat or Number needed to harm are ways of expressing the effectiveness and safety of an intervention in a way that is clinically meaningful. In general, NNT is always computed with respect to two treatments A and B, with A typically a drug and B a placebo (in our example above, A is a 5-year treatment with the hypothetical drug, and B is no treatment). A defined endpoint has to be specified (in our example: the appearance of colon cancer in the 5 year period). If the probabilities pA and pB of this endpoint under treatments A and B, respectively, are known, then the NNT is computed as 1/(pB-pA). If it is said that the NNT for breast mammography is 285 this means that 285 mammograms need to be performed to diagnose one case of breast cancer. As another example, an NNT of 4 means if 4 patients are treated, only one would respond.
An NNT of 1 is the most effective and means each patient treated responds, e.g., in comparing antibiotics with placebo in the eradication of Helicobacter pylori. An NNT of 2 or 3 indicates that a treatment is quite effective (with one patient in 2 or 3 responding to the treatment). An NNT of 20 to 40 can still be considered clinically effective.
Quality of clinical trials
Evidence-based medicine attempts to objectively evaluate the quality of clinical research by critically assessing techniques reported by researchers in their publications.
- Trial design considerations. High-quality studies have clearly defined eligibility criteria and have minimal missing data.
- Generalizability considerations. Studies may only be applicable to narrowly defined patient populations and may not be generalizable to other clinical contexts.
- Follow-up. Sufficient time for defined outcomes to occur can influence the study outcomes and the statistical power of a study to detect differences between a treatment and control arm.
- Power. A mathematical calculation can determine if the number of patients is sufficient to detect a difference between treatment arms. A negative study may reflect a lack of benefit, or simply a lack of sufficient quantities of patients to detect a difference.
Limitations and criticism
Although evidence-based medicine is regarded as the gold standard of conventional clinical practice, there are a number of limitations and criticisms of its use, many of which remain unresolved despite nearly two centuries of debate.
- EBM produces quantitative research, especially from randomized controlled trials (RCTs). Accordingly, results may not be relevant for all treatment situations.
- RCTs are expensive, influencing research topics according to the sponsor's interests.
- There is a lag between when the RCT is conducted and when its results are published.
- There is a lag between when results are published and when these are properly applied.
- Certain population segments have been historically under-researched (racial minorities and people with co-morbid diseases), and thus the RCT restricts generalizing.
- Not all evidence from an RCT is made accessible. Treatment effectiveness reported from RCTs may be different than that achieved in routine clinical practice.
- Published studies may not be representative of all studies completed on a given topic (published and unpublished) or may be unreliable due to the different study conditions and variables.
- EBM applies to groups of people but this does not preclude clinicians from using their personal experience in deciding how to treat each patient. One author advises that "the knowledge gained from clinical research does not directly answer the primary clinical question of what is best for the patient at hand" and suggests that evidence-based medicine should not discount the value of clinical experience. Another author stated that "the practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research".
- Hypocognition (the absence of a simple, consolidated mental framework that new information can be placed into) can hinder the application of EBM.
Assessing the teaching of evidence-based medicine
Standard descriptions about mental illnesses, such as the Diagnostic and Statistical Manual of Mental Disorders (DSM), have been criticized as incompletely justified by evidence. In many cases, it is unknown whether a particular "disease" has one, several, or no underlying biological causes, with controversy arising over whether some diseases are merely an artifact of the attempt to construct a unified classification scheme, rather than a "real" disease.
While some experts point to statistics in support of the idea that a lack of adoption of research findings results in suboptimal treatment for many patients, others emphasize the importance of the skill of the practitioner and the customization of the treatment to fit individual needs. There is some controversy over whether mental illnesses are too complex for broad population studies to be helpful.
Traces of evidence-based medicine's origin can be found in ancient Greece. Although testing medical interventions for efficacy has existed since the time of Avicenna's The Canon of Medicine in the 11th century, it was only in the 20th century that this effort evolved to impact almost all fields of health care and policy.
In 1967, the American physician and mathematician Alvan R. Feinstein published his groundbreaking work Clinical Judgment, which together with Archie Cochrane’s famous book Effectiveness and Efficiency (1972) led to an increasing acceptance of clinical epidemiology and controlled studies during the 70s and 80s and paved the way for the institutional development of EBM in the 90s. Cochrane's efforts were recognized by the fact that an international network for efficacy assessment in medicine - the Cochrane Collaboration – was posthumously named after him. However, Cochrane himself did not live to see the foundation and institutionalization of the EBM movement and Feinstein became later in his life one of its sharpest methodological critics.
The explicit methodologies used to determine "best evidence" were largely established by the McMaster University research group led by David Sackett and Gordon Guyatt. Guyatt later coined the term "evidence-based" in 1990. The term "evidence-based medicine" first appeared in the medical literature in 1992 in a paper by Guyatt et al. Relevant journals include the British Medical Journal's Clinical Evidence, the Journal of Evidence-Based Healthcare and Evidence Based Health Policy. All of these were co-founded by Anna Donald, an Australian pioneer in the discipline.
EBM and ethics of experimental or risky treatments
Insurance companies in the United States and public insurers in other countries usually wait for drug use approval based on evidence-based guidelines before funding a treatment. Where approval for a drug has been given, and subsequent evidence-based findings indicating that a drug may be less safe than originally anticipated, some insurers in the U.S. have reacted very cautiously and withdrawn funding. For example, an older generic statin drug had been shown to reduce mortality, but a newer and much more expensive statin drug was found to lower cholesterol more effectively. However, evidence came to light about safety concerns with the new drug which caused some insurers to stop funding it even though marketing approval was not withdrawn.
Some people are willing to take their chances to gamble their health on the success of new drugs or old drugs in new situations which may not yet have been fully tested in clinical trials. However insurance companies are reluctant to take on the job of funding such treatments, preferring instead to take the safer route of awaiting the results of clinical testing and leaving the funding of such trials to the manufacturer seeking a license.
Sometimes caution errs in the other direction. Kaiser Permanente did not change its methods of evaluating whether or not new therapies were too "experimental" to be covered until it was successfully sued twice: once for delaying in vitro fertilization treatments for two years after the courts determined that scientific evidence of efficacy and safety had reached the "reasonable" stage; and in another case where Kaiser refused to pay for liver transplantation in infants when it had already been shown to be effective in adults, on the basis that use in infants was still "experimental." Here again, the problem of induction plays a key role in arguments.
Application of the evidence based model on other public policy matters
There has been discussion of applying what has been learned from EBM to public policy. In his 1996 inaugural speech as President of the Royal Statistical Society, Adrian Smith held out evidence-based medicine as an exemplar for all public policy. He proposed that "evidence-based policy" should be established for education, prisons and policing policy and all areas of government.
- Clinical decision support system (CDSS)
- Consensus (medical)
- Evidence-based design
- Evidence-based management
- Evidence-based nursing
- Policy-based evidence making
- Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (January 1996). "Evidence based medicine: what it is and what it isn't". BMJ 312 (7023): 71–2. doi:10.1136/bmj.312.7023.71. PMC 2349778. PMID 8555924.
- Timmermans S, Mauck A (2005). "The promises and pitfalls of evidence-based medicine". Health Aff (Millwood) 24 (1): 18–28. doi:10.1377/hlthaff.24.1.18. PMID 15647212.
- Greenhalgh, Trisha. How To Read a Paper: The Basics of Evidence-Based Medicine. Wiley-Blackwell, fourth edition, 2010, p. 1.
- Tonelli, MR (2001 Dec). "The limits of evidence-based medicine.". Respiratory care 46 (12): 1435–40; discussion 1440–1. PMID 11728302.
- Katz 2001, pp. xi–xiv
- Grobbee & Hoes 2009, pp. 3–6
- Doi 2012
- Evidence-Based Medicine Working Group (November 1992). "Evidence-based medicine. A new approach to teaching the practice of medicine". JAMA 268 (17): 2420–5. doi:10.1001/jama.268.17.2420. PMID 1404801.
- "Evidence-based health care and systematic reviews". The Cochrane Collaboration. Retrieved 13 September 2012.
- "What is EBM?". CEBM – Centre for Evidence Based Medicine. Retrieved 13 September 2012.
- Gray, J. A. Muir (2009). Evidence-based health care & public health. Edinburgh: Churchill Livingstone. ISBN 9780443101236.
- Fischer, Michael Daniel; Ferlie, Ewan (1 January 2013). "Resisting hybridisation between modes of clinical risk management: Contradiction, contest, and the production of intractable conflict". Accounting, Organizations and Society 38 (1): 30–49. doi:10.1016/j.aos.2012.11.002.
- Dopson et al (2013). "Health care managers' access and use of management knowledge". National Institute of Health Research, Service Delivery & Organisation Programme. Retrieved 4 June 2013.
- Ferlie, E. & McGivern, G. (2013) Bringing Anglo-Governmentality into Public Management Scholarship: The Case of Evidence-Based Medicine in the UK. Journal of Public Administration Research & Theory. First published online: February 27, 2013
- Eddy DM (2005). "Evidence-based medicine: a unified approach". Health Aff (Millwood) 24 (1): 9–17. doi:10.1377/hlthaff.24.1.9. PMID 15647211.
- Cook DJ, Jaeschke R, Guyatt GH (1992). "Critical appraisal of therapeutic interventions in the intensive care unit: human monoclonal antibody treatment in sepsis. Journal Club of the Hamilton Regional Critical Care Group". J Intensive Care Med 7 (6): 275–82. PMID 10147956.
- Dawes M, Summerskill W, Glasziou P, et al. (January 2005). "Sicily statement on evidence-based practice". BMC Med Educ 5 (1): 1. doi:10.1186/1472-6920-5-1. PMC 544887. PMID 15634359. Archived from the original on 2011-02-17.
- Richardson WS, Wilson MC, Nishikawa J, Hayward RS (1995). "The well-built clinical question: a key to evidence-based decisions". ACP J. Club 123 (3): A12–3. PMID 7582737.
- Rosenberg WM, Deeks J, Lusher A, Snowball R, Dooley G, Sackett D (1998). "Improving searching skills and evidence retrieval". J R Coll Physicians Lond 32 (6): 557–63. PMID 9881313.
- Parkes J, Hyde C, Deeks J, Milne R (2001). "Teaching critical appraisal skills in health care settings". In Santesso, Nancy. Cochrane Database Syst Rev (3): CD001270. doi:10.1002/14651858.CD001270. PMID 11686986.
- Epling J, Smucny J, Patil A, Tudiver F (October 2002). "Teaching evidence-based medicine skills through a residency-developed guideline". Fam Med 34 (9): 646–8. PMID 12455246.
- Jamtvedt G, Young JM, Kristoffersen DT, Thomson O'Brien MA, Oxman AD (2003). "Audit and feedback: effects on professional practice and health care outcomes". In Jamtvedt, Gro. Cochrane Database Syst Rev (3): CD000259. doi:10.1002/14651858.CD000259. PMID 12917891.
- Tanjong-Ghogomu, E; Tugwell, P; Welch, V (2009). "Evidence-based medicine and the Cochrane Collaboration". Bulletin of the NYU hospital for joint diseases 67 (2): 198–205. PMID 19583554.
- El Dib RP, Atallah AN, Andriolo RB (August 2007). "Mapping the Cochrane evidence for decision making in health care". J Eval Clin Pract 13 (4): 689–92. doi:10.1111/j.1365-2753.2007.00886.x. PMID 17683315.
- Ezzo J, Bausell B, Moerman DE, Berman B, Hadhazy V (2001). "Reviewing the reviews. How strong is the evidence? How clear are the conclusions?". Int J Technol Assess Health Care 17 (4): 457–466. PMID 11758290.
- Committee on the Use of Complementary and Alternative Medicine by the American Public (2005). "5 State of Emerging Evidence on CAM: Cochrane Review Evidence for CAM". Complementary and Alternative Medicine in the United States. Washington DC: The National Academies Press. pp. 135–6. ISBN 978-0-309-09270-8.
- "EBM: Levels of Evidence". Essential Evidence Plus. Retrieved 2012-02-23.
- Tonelli MR (November 1999). "In defense of expert opinion". Acad Med 74 (11): 1187–92. doi:10.1097/00001888-199911000-00010. PMID 10587679.
- U.S. Preventive Services Task Force (August 1989). Guide to clinical preventive services: report of the U.S. Preventive Services Task Force. DIANE Publishing. pp. 24–. ISBN 978-1-56806-297-6.
- "Levels of Evidence". CEBM. Archived from the original on 2008-04-10.
- "Task Force Ratings". Retrieved 2007-09-24.
- "GRADE working group". Archived from the original on 2006-04-07. Retrieved 2007-09-24.
- Balshem H, Helfand M, Schünemann HJ, et al. (April 2011). "GRADE guidelines: 3. Rating the quality of evidence". J Clin Epidemiol 64 (4): 401–6. doi:10.1016/j.jclinepi.2010.07.015. PMID 21208779.
- Schünemann H, Brożek J, Oxman A,, ed. (2009). [Available from http://www.cc-ims.net/gradepro GRADE handbook for grading quality of evidence and strength of recommendation] (Version 3.2 ed.).[dead link]
- McQuay, Henry J.; Moore, R. Andrew (1997-05-01). "Numbers Needed to Treat". Bandolier. Retrieved 2006-06-27.
- http://ije.oupjournals.org/cgi/content/full/30/6/1246 Statistical research on conditions caused by calculi by Doctor Civiale. 1835.] Int J Epidemiol. 2001 Dec;30(6):1246-9. Reprint of the classical report by Poisson, Dulong, Larrey, Double.
- Upshur RE, VanDenKerkhof EG, Goel V (May 2001). "Meaning and measurement: an inclusive model of evidence in health care". J Eval Clin Pract 7 (2): 91–6. doi:10.1046/j.1365-2753.2001.00279.x. ISSN 1356-1294. PMID 11489034.
- Yitschaky O, Yitschaky M, Zadik Y (May 2011). "Case report on trial: Do you, Doctor, swear to tell the truth, the whole truth and nothing but the truth?" (PDF). J Med Case Reports 5 (1): 179. doi:10.1186/1752-1947-5-179. PMC 3113995. PMID 21569508.
- "Knowledge Transfer in the ED: How to Get Evidence Used". Best Evidence Healthcare Blog. Retrieved 8 October 2013.
- Rogers WA (April 2004). "Evidence based medicine and justice: a framework for looking at the impact of EBM upon vulnerable or disadvantaged groups". J Med Ethics 30 (2): 141–5. doi:10.1136/jme.2003.007062. PMC 1733835. PMID 15082806.
- "Patient Compliance with statins" Bandolier Review 2004
- Friedman LS, Richter ED (January 2004). "Relationship Between Conflicts of Interest and Research Results". J Gen Intern Med 19 (1): 51–6. doi:10.1111/j.1525-1497.2004.30617.x. PMC 1494677. PMID 14748860.
- Mariotto, A. (2010). "Hypocognition and evidence-based medicine". Internal Medicine Journal 40 (1): 80–82. doi:10.1111/j.1445-5994.2009.02086.x. PMID 20561370.
- Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH, Kunz R (2002). "Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine.". BMJ (Clinical research ed.) 325 (7376): 1338–41. PMC 137813. PMID 12468485.
- Ramos KD, Schafer S, Tracz SM (2003). "Validation of the Fresno test of competence in evidence based medicine.". BMJ 326 (7384): 319–21. doi:10.1136/bmj.326.7384.319. PMC 143529. PMID 12574047.
- Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S et al. (2006). "Instruments for evaluating education in evidence-based practice: a systematic review.". JAMA 296 (9): 1116–27. doi:10.1001/jama.296.9.1116. PMID 16954491.
- Straus SE, Green ML, Bell DS, Badgett R, Davis D, Gerrity M et al. (2004). "Evaluating the teaching of evidence based medicine: conceptual framework.". BMJ 329 (7473): 1029–32. doi:10.1136/bmj.329.7473.1029. PMC 524561. PMID 15514352.
- Kunz R, Wegscheider K, Fritsche L, Schünemann HJ, Moyer V, Miller D et al. (2010). "Determinants of knowledge gain in evidence-based medicine short courses: an international assessment.". Open Med 4 (1): e3–e10. PMC 3116678. PMID 21686291.
- West CP, Jaeger TM, McDonald FS (2011). "Extended evaluation of a longitudinal medical school evidence-based medicine curriculum.". J Gen Intern Med 26 (6): 611–5. doi:10.1007/s11606-011-1642-8. PMC 3101983. PMID 21286836.
- Sobo S (May 2009). "Pursuing treatments that are not evidence based: how DSM IV clarifies, how it blinds psychiatrists to issues in need of investigation". Med. Hypotheses 72 (5): 491–8. doi:10.1016/j.mehy.2008.12.022. PMID 19181456.
- "Can Science Make Psychotherapy More Effective?". Science Friday / National Public Radio. Retrieved 2010-02-09.
- Woolf SH, George JN (August 2000). "Evidence-based medicine. Interpreting studies and setting policy". Hematol. Oncol. Clin. North Am. 14 (4): 761–84. doi:10.1016/S0889-8588(05)70310-5. PMID 10949772.
- Brater DC, Daly WJ (May 2000). "Clinical pharmacology in the Middle Ages: principles that presage the 21st century". Clin. Pharmacol. Ther. 67 (5): 447–50. doi:10.1067/mcp.2000.106465. PMID 10824622. "p. 449"
- Daly WJ, Brater DC (2000). "Medieval contributions to the search for truth in clinical medicine". Perspect. Biol. Med. 43 (4): 530–40. doi:10.1353/pbm.2000.0037. ISSN 0031-5982. PMID 11058989. "p. 536"
- Heinrich Weßling (2011-04). Theorie der klinischen Evidenz: Versuch einer Kritik der Evidenzbasierten Medizin. LIT Verlag Münster. pp. 39–50. ISBN 978-3-643-90065-4.
- Jeremy H. Howick (2011-02-23). The Philosophy of Evidence-based Medicine. John Wiley & Sons. p. 15. ISBN 978-1-4443-4266-6.
- Appleby, Julie (2004-12-26). "What do you believe when drug messages conflict?". USA Today. Retrieved 2010-05-02.
- Colliver, Victoria (2006-02-12). "In fight for life, insurer no help". The San Francisco Chronicle.
- Sugarman M (Winter 2001). "Permanente Physicians Determine Use of New Technology: Kaiser Permanente's Interregional New Technologies Committee". The Permanente Journal 5 (1).
- Smith, A.F.M. (1996). "Mad cows and ecstasy: chance and choice in an evidence-based society". Journal of the Royal Statistical Society, Series A 159 (3): 367–83. doi:10.2307/2983324.
- Doi, S.A.R. (2012). Understanding evidence in health care: Using clinical epidemiology. South Yarra, VIC, Australia: Palgrave Macmillan. ISBN 978-1-4202-5669-7.
- Grobbee, D.E.; Hoes, Arno W. (2009). Clinical Epidemiology: Principles, Methods, and Applications for Clinical Research. Jones & Bartlett Learning. ISBN 978-0-7637-5315-3.
- Howick, Jeremy H. (2011). The Philosophy of Evidence-based Medicine. Wiley. ISBN 978-1-4051-9667-3.
- Katz, David L. (2001). Clinical Epidemiology & Evidence-Based Medicine: Fundamental Principles of Clinical Reasoning & Research. SAGE. ISBN 978-0-7619-1939-1.
|Look up evidence-based medicine in Wiktionary, the free dictionary.|