Dodo bird verdict
The Dodo bird verdict (or Dodo bird conjecture) is a controversial topic in psychotherapy, referring to the claim that all psychotherapies, regardless of their specific components, produce equivalent outcomes. The conjecture was introduced by Saul Rosenzweig in 1936, drawing on imagery from Lewis Carroll's novel Alice's Adventures in Wonderland, but only came into prominence with the emergence of new research evidence in the 1970s.
The importance of the continuing debate surrounding the Dodo bird verdict stems from its implications for professionals involved in the field of psychotherapy and the psychotherapies made available to clients.
The Dodo bird verdict terminology was coined by Saul Rosenzweig in 1936 to illustrate the notion that all therapies are equally effective. Rosenzweig borrowed the phrase from Carroll, Lewis (1865), Alice's Adventures in Wonderland, wherein a number of characters become wet and, in order to dry themselves, the Dodo Bird decided to issue a competition: everyone was to run around the lake until they were dry. Nobody cared to measure how far each person had run, nor how long. When they asked the Dodo who had won, he thought long and hard and then said "Everybody has won and all must have prizes." In the case of psychotherapies, Rosenzweig argued that common factors were more important than specific technical differences, so that (on the Dodo bird conjecture) all therapies are winners; they all produce equally effective outcomes.
The Dodo bird debate only took flight in 1975 when Lester Luborsky, Barton Singer and Lise Luborsky reported the results of one of the first comparative studies demonstrating few significant differences in the outcomes among different psychotherapies. This study spurred a plethora of new studies in both opposition to and support of the Dodo bird verdict.
The Dodo bird debate, in brief, is focused on whether or not the specific components of different treatments lead some treatments to outperform other treatments for specific disorders. Supporters of the Dodo bird verdict contend that all psychotherapies are equivalent because of "common factors" that are shared in all treatments (i.e., having a relationship with a therapist who is warm, respectful, and has high expectations for client success). In contrast, critics of the Dodo bird verdict would argue that the specific techniques used in different therapies are important, and all therapies do not produce equivalent outcomes for specific disorders.
The common factors theory states that if all therapies in psychology are equally effective, it is because of the common factors they share. The most important causal agents in treatment are the common factors; the specific techniques that are unique to treatment strategies have only minor importance.
There is research to support the common factors theory. One common factor is the client–therapist interaction, also known as the therapeutic alliance. A 1992 paper by Lambert showed that nearly 40 percent of the improvement in psychotherapy is from these client–therapist variables. Other researchers have further analyzed the importance of client–therapist variables in treatment. They found that improvement in the patient was due to extra-therapeutic factors, for example patients' thought processes. Data shows that patients with more positive attitudes will have a better chance of experiencing clinical improvement, regardless of the therapist's actions. Furthermore, in a meta-analysis of many studies of psychotherapy, Wampold et al. 2002, found that 7% of the variability in treatment outcome was due to the therapeutic alliance whereas 1% of the variability was due to a specific treatment. The therapist's attitude is also a very important causal agent in positive patient change. Najavits and Strupp (1994) demonstrated that a positive, warm, caring, and genuine therapist generated statistically significant differences in patient outcome. Wampold et al. 2002, also found that nearly 70% of the variability in treatment outcome was due to the therapist's attitude toward the efficacy of the treatment. Specific components of therapy are concluded to be relatively frivolous when compared with the more profound and directly patient affecting common factors.
Researchers have studied common factors in detail. Grencavage and Norcross (1990) identified 35 common factors in published sources. The identified common factors were categorized into five main groups: client characteristics, therapist qualities, change processes, treatment structures and relationship elements. Examples of some of the common factors included within these broad categories are persuasion, a healing setting, engagement, the use of rituals and techniques, suggestion, and emotional learning. Recently, Tracey et al. 2003, examined deeper relationships among the categories and common factors. They concluded that there are two dimensions of therapy: feeling and thinking. Within each of the two dimensions are three clusters: bond, information, and role.
Data provide evidence for the Dodo bird verdict. Generally speaking, common factors are responsible for patient improvement instead of any component of a specific treatment. Researchers such as Wampold and Luborsky continue to support this claim in recent articles published in the Behavioral Therapist. Wampold et al. 2010 refutes claims made by Siev et al. 2009 that Wampold 2009 made errors in research. Wampold et al. 2009, suggests that people need to "accept the importance of the alliance and therapists and remain committed to developing and improving treatments." Wampold continues by saying that techniques could be beneficial in psychotherapy because they are the easiest variables to manipulate. These variables can act to change alliance and other common factors. Common factors can then be closely monitored and carefully distributed to the patient via the therapist.
In opposition to the Dodo bird verdict, there are a growing number of studies demonstrating that some treatments produce better outcomes for particular disorders when compared to other treatments. Here, in contrast to the common factor theory, specific components of the therapy have shown predictive power. The most compelling evidence against the Dodo bird verdict is illustrated by the research done on anxiety disorders. Many studies have found specific treatment modalities to be beneficial when treating anxiety disorders, specifically cognitive behavioral therapy (CBT). CBT uses techniques from both cognitive therapy and behavioral therapy to modify maladaptive thoughts and behaviors. Numerous meta-analyses have shown that CBT yields significantly superior results in the treatment of psychological disorders, most notably, anxiety disorders. However, CBT also plays a positive role in treating depression, eating disorders, substance abuse disorders, and obsessive-compulsive disorder. In meta-analytic reviews, there is generally a larger effect size when CBT is used to treat anxiety disorders. Recent studies show that when treating generalized anxiety disorder, CBT generated an effect size of 0.51, which indicates a medium effect. That is a much larger effect compared to supportive therapy and other treatments. Similarly, when treating social anxiety disorder, CBT produced an effect size of 0.62, which again supports the evidence that CBT does in fact yield significantly better results than other therapies. Those supporting the Dodo bird verdict often use meta-analyses to compare multiple evidence supported treatments (ESTs) in order to illustrate that these treatments have no really significant differences. Because these treatments are already proven to work, comparing them to each other does show little variation. However psychology is made up of more therapies than just ESTs. When comparing a therapy like CBT to a treatment that does not meet EST criteria, the difference in effect size is obvious. Research showing differences among different treatments for specific disorders is now reflected in the latest EST guidelines. ESTs are developed from multiple control trial tests to evaluate which therapies yield the best results for specific disorders. These disorders include, but are not limited to, major depressive disorder, anxiety disorder, panic disorder, and OCD. According to the Ethics Code of the American Psychological Association (APA), psychologists and therapists have an obligation to avoid harming their clients in any way. ESTs are a major component in this movement. By using specific therapies clinically proven to improve certain disorders, therapists are avoiding further damage to their patients. However, supporting the idea of ESTs inevitably implies that some therapies are in fact more efficacious than others for particular disorders.
There is also research suggesting that some treatments might be harmful. Indeed, "if psychotherapy is powerful enough to do good, it may be powerful enough to do harm," as well. Some psychotherapies can be labeled as unhelpful, while others fall under the category of harmful. Unhelpful treatments are those that give no assistance while harmful therapies are actually damaging or dangerous to the patient. When identifying a "harmful" treatment it is important to note the distinction between "harm that can be caused by a disorder and harm that can be caused by the application of a treatment," The negative outcomes of some psychotherapies are becoming an alarming reality in the field of clinical psychology. Studies have shown that individuals exhibited negative responses to treatment in some substance abuse work, and some types of grief therapy, and certain therapeutic techniques with trauma and PTSD patients. While those studies that support the Dodo bird verdict focus on the importance of building a client–therapist relationship, some studies have "identified a number of other relationship factors that may interfere with or negatively impact therapeutic change". The emerging evidence to the effect that there are possibly harmful psychotherapies is not only contradictory to the "all therapies are equal" stance of the Dodo bird verdict, but may also point out problems implicating the APA's Code of Ethics. Many meta-analyses show that there are treatments that do yield more positive outcomes for specific disorders. However, proof that some treatments actually harm patients instead of helping them further erodes support for the Dodo bird verdict.
Issues in testing
A considerable amount of the controversy about the Dodo bird verdict relates to the meta-analytic methods used in research. These generate a lack of clear psychotherapeutic evidence for the support of either side in the debate. Meta-analytic studies have compared the effect sizes of different treatments, but have not been reliable in finding a consistent effect size. This could be because of several confounding variables. For example, many researchers are said to "have an agenda" when conducting meta-analyses, cherry-picking experiments they want to use in their study in order to produce the results they want. This pre-determined skewing results in unfair representations of the research. "Agenda"-based meta-analyses are confounded with the researcher's political, social, and economical opinions. Since psychologists are given the power to choose which studies are used in a meta-analysis, personal biases are involved, and the meta-analysis will produce biased results if the researcher is not careful in controlling for his or her own opinions.
Researchers opposing the Dodo bird verdict have found Dodo bird supporters' meta-analyses to be "agenda"-based and highly subjective. Arguments have proposed that meta-analyses could possibly produce misleading results because of the type of studies combined in the comparison. In Paul Crits-Christoph's review of Wampold et al.'s (1997) comparative study, a work that supported the Dodo bird verdict, he concluded that out of the 114 articles used in the study, 79 of them involve similar comparative tests. Some meta-analyses constructed are not sensitive to the subtle distinctions between treatment effects, especially among comparative studies of highly similar treatments.
Dodo bird supporting researchers have found anti-Dodo bird supporters' research to also be "agenda"-based. For example, Wampold (2009) found Siev et al.'s (2007) study whose research for significance of CBT (Cognitive Behavior Therapy) versus RT (Rational Therapy) was resting on one experiment with an uncharacteristically large effect size (1.02) by Clark et al. (1994). Wampold found this effect size to be invalid because of the internal biases of the study. When this flawed experiment was removed from the analysis, the effect size was not statistically significant for the use of CBT over RT in panic disorder therapy. Against this research, in support of the anti-Dodo bird verdict, Chambless (2002) stated that "errors in data analysis, exclusion of research on many types of clients, faulty generalization to comparisons between therapies that have never been made, and erroneous sorts of treatments for all sorts of problems can be assumed to represent the difference between any two types of treatment for a given problem."
Clearly, if meta-analyses are to be taken seriously, they need to be conducted carefully and objectively. In support of the anti-Dodo bird side, Hunsley (2007) says that when "measurement quality is controlled for and when treatments are appropriately categorized, there is consistent evidence in both treatment outcome and comparative treatment research that cognitive and behavioral treatments are superior to other treatments for a wide range of conditions, in both adult and child samples." This suggests that if and when variables are controlled, there is appreciable evidence for the superiority of cognitive and behavioral treatments.
The outcome of the Dodo bird debate has extremely important implications for the future of clinical psychology. For one, policymakers have to know how effective each existing psychotherapy is in order to be able to decide which therapies should be supported. This controversy may also lead governmental officials to cut funding for psychological treatments and other public health measures
Perhaps the greatest illustration of the current state of the Dodo bird verdict is seen in the meta-analyses of Wampold and Barlow and the responses to it. In these meta-analyses, researchers on both sides point out the weaknesses and inconsistencies in their opponents' positions. Although both sides are trying to improve psychology in their respective ways, the disagreement about and lack of consistent evidence for the Dodo bird verdict may in fact be the cause of increased public doubt about the field. The conclusion of the debate could nationally dictate which therapists and which procedures will remain financially supported. For example, if the Dodo bird verdict is thought to be true regarding different psychotherapies, then many clinicians would feel free to use any therapy they see fit to employ. However, if the Dodo bird verdict is proven to be false, then clinicians would likely have to use empirically supported therapies when treating their clients.
Fuelling the debate have been alternative visions or paradigms of mental illness. Those believing in a medical model of mental illness and cure see the Dodo bird verdict as necessarily untrue – even absurd – whatever the evidence supporting it. Those who see therapy as context-based – as relying on a shared frame of reference or context between client and therapist for optimum results – will almost equally automatically welcome the Dodo bird verdict.
- Bentall, Richard P (2009), Doctoring the mind: is our current treatment of mental illness really any good?, New York: New York University Press, pp. 248–249.
- Rosenzweig, Saul (1936). "Some implicit common factors in diverse methods of psychotherapy". American Journal of Orthopsychiatry 6 (3): 412–15. doi:10.1111/j.1939-0025.1936.tb05248.x.
- Luborsky, L (1999). "The researcher's own therapeutic allegiances: a 'wild card' in comparisons of treatment efficacy". Clinical Psychology: Science and Practice 6: 49–62.
- Luborsky, L; Singer, B; Luborsky, L (1975), "Is it true that 'everyone has won and all must have prizes?'", Archives of General Psychiatry 32: 995–1008, doi:10.1001/archpsyc.1975.01760260059004.
- Luborsky, L; Rosenthal, R; Diguer, L; Andrusyna, TP; Berman, JS; Levitt, JT; Seligman, DA; Krause, ED (2002), "The dodo bird verdict is alive and well–mostly", Clinical Psychology: Science and Practice 9: 2–12, doi:10.1093/clipsy/9.1.2.
- Tarrier, N (2002), "Yes, cognitive behaviour therapy may be all you need", British Medical Journal 324: 291–92.
- Frank, JD (1961), Persuasion and healing: A comparative study of psychotherapy.
- Wampold, Bruce E (2007), "Psychotherapy: the humanistic (and effective) treatment", American Psychologist 62: 857–73, doi:10.1037/0003-066x.62.8.857.
- Wampold, Bruce E (2001), The great psychotherapy debate: models, methods, and findings, Mahwah, NJ: L. Erlbaum, pp. 36, 96.
- Horvath, AO; Bedi, RP (2002), "The alliance", in Norcross, JC, Psychotherapy relationships that work: therapist contributions and responsiveness to patients, New York: Oxford University Press, pp. 37–70.
- Martin, DJ; Garske, JP; Davis, MK (2000), "Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review", Journal of Consulting and Clinical Psychology: 438–50, doi:10.1037/0022-006x.68.3.438.
- Baldwin, SA; Wampold, Bruce E; Imel, ZE (2007), "Untangling the alliance–outcome correlation: exploring the relative importance of therapist and patient variability in the alliance", Journal of Consulting and Clinical Psychology 75: 842–52, doi:10.1037/0022-006x.75.6.842.
- Klein, DN; Schwartz, JE; Santiago, NJ; Vivian, D; Vocisano, C; Castonguay, LG et al. (2003), "Therapeutic alliance in depression treatment: Controlling for prior change and patient characteristics", Journal of Consulting and Clinical Psychology 71: 997–1006, doi:10.1037/0022-006x.71.6.997.
- Lambert, M (1992), "Implications for outcome research for psychotherapy integration", in Norcross, JC; Goldstein, MR, Handbook of psychotherapy integration, New York: Basic Books, pp. 94–129.
- Wampold, Bruce E (1997), "Methodological problems in identifying efficacious psychotherapies", Psychotherapy Research 7: 21–43, doi:10.1080/10503309712331331853.
- Wampold, Bruce E; Minami, T; Baskin, TW; Tierney, SC (2002), "A meta-(re)analysis of the effects of cognitive therapy versus 'other therapies' for depression", Journal of Affective Disorders 68: 159–65, doi:10.1016/s0165-0327(00)00287-1.
- Wampold, Bruce E; Mondin, GW; Moody, M; Ahn, H (1997), "The flat earth as a metaphor for the evidence for uniform efficacy of bona fide psychotherapies: reply to Crits-Christoph (1997) and Howard et al. (1997)", Psychological Bulletin 122: 226–30, doi:10.1037/0033-2909.122.3.226.
- Wampold, Bruce E; Mondin, GW; Moody, M; Stich, F; Benson, K; Ahn, H (1997), "A meta-analysis of outcome studies comparing bona fide psychotherapies: empirically, 'All must have prizes'", Psychological Bulletin 122: 203–15, doi:10.1037/0033-2909.122.3.203.
- Wampold, Bruce E; Serlin, RC (2000), "The consequences of ignoring a nested factor on measures of effect size in analysis of variance", Psychological Methods 5: 425–33, doi:10.1037/1082-989x.5.4.425.
- Najavits, LM; Strupp, HH (1994), "Differences in the effectiveness of psychodynamic therapies: a process-outcome study", Psychotherapy 31 (1): 114–23, doi:10.1037/0033-3126.96.36.199.
- Grencavage, Lisa M; Norcross, John C (1990). "Where are the commonalities among the therapeutic common factors?". Professional Psychology: Research and Practice 21 (5): 372–378. doi:10.1037/0735-7028.21.5.372.
- Tracey, Terence JG (2003), "Concept mapping of therapeutic common factors", Psychotherapy Research 13 (4): 401–13, doi:10.1093/ptr/kpg041.
- Wampold, BE; Imel, ZE; Miller, D (2009), "Barriers to the dissemination of empirically supported treatments: Matching evidence to messages", The Behavior Therapist 32: 144–55.
- Siev, J; Chambless, DL (2007), "Specificity of treatment effects: Cognitive therapy and re- laxation for generalized anxiety and panic disorders", Journal of Consulting and Clinical Psychology 75: 513–22, doi:10.1037/0022-006x.75.4.513.
- Siev, J; Chambless, DL (2009). "The dodo bird, treatment technique, and disseminating empirically supported treatments". The Behavior Therapist 32: 69–75.
- DeRubeis, RJ; Brotman, MA; Gibbons, CJ (2005). "A conceptual and methodological analysis of the nonspecifics argument". Clinical Psychology: Science and Practice 12: 174–83. doi:10.1093/clipsy.bpi022.
- Clark, DM; Ehlers, A; Hackmann, A; McManus, F; Fennell, M; Grey, N (2006), et al., "Cognitive therapy versus exposure and applied relaxation in social phobia: A randomized controlled trial", Journal of Consulting and Clinical Psychology 74: 568–78, doi:10.1037/0022-006x.74.3.568.
- Chambless, Dianne L; Hollon, SD (1998), "Defining empirically supported therapies", Journal of Consulting and Clinical Psychology 66: 7–18, doi:10.1037/0022-006x.66.1.7.
- Siev, Jedidiah; Huppert, Jonathan; Chambless, Dianne (Jan 2010), "Treatment specificity for panic disorder: a reply to Wampold, Imel, and Miller (2009)", The Behavior Therapist 33 (1): 12–14.
- Watson, D; O'Hara, MW; Stuart, S (2008). "Hierarchical structures of affect and psychopathology and their implications for the classification of emotional disorders". Anxiety and Depression 25: 282–88. doi:10.1002/da.20496.
- Arch, JJ; Craske, MG (2009). "First-line treatment: a critical appraisal of cognitive behavioral therapy developments and alternatives". Psychiatric Clinics of North America 32: 525–47. doi:10.1016/j.psc.2009.05.001.
- Hassett, A; Gevirtz, R. "Nonpharmacologic treatment for fibromyalgia: Patient education, cognitive-behavioral therapy, relaxation techniques, and complementary and alternative medicine". Rheumatic Disease Clinics of North America 35: 393–407. doi:10.1016/j.rdc.2009.05.003. PMC 2743408. PMID 19647150.
- Beck, AT (1964). "Thinking and depression: theory and therapy". Archives of General Psychiatry 10: 561–71. doi:10.1001/archpsyc.1964.01720240015003.
- Butler, AC; Chapman, JE; Forman, EM; Beck, AT (2006), "The empirical status of cognitive-behavioral therapy: a review of meta-analyses", Clinical Psychology Review 26: 17–31, doi:10.1016/j.cpr.2005.07.003, PMID 16199119
- Hofmann, SG; Smits, JA (2008). "Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials". Journal of Clinical Psychiatry 69: 621–32.
- Otte, C (2011). "Cognitive behavioral therapy in anxiety disorders: Current state of the evidence". Dialogues in Clinical Neuroscience 13: 413–21.
- Driessen, E; Hollon, S.D. (2010). "Cognitive behavioral therapy for mood disorders: efficacy, moderators and mediators". Psychiatric Clinics of North America 33: 537–55. doi:10.1016/j.psc.2010.04.005. PMC 2933381. PMID 20599132.
- Murphy, R; Straebler, S; Cooper, Z; Fairburn, CG (2010). "Cognitive behavioral therapy for eating disorders". Psychiatric Clinics of North America 33: 611–27. doi:10.1016/j.psc.2010.04.004. PMC 2928448. PMID 20599136.
- McHugh, RK; Hearon, BA; Otto, MW (2010). "Cognitive behavioral therapy for substance use disorders". Psychiatric Clinics of North America 33: 511–25. doi:10.1016/j.psc.2010.04.012. PMC 2897895. PMID 20599130.
- Boileau, B (2011). "A review of obsessive-compulsive disorder in children and adolescents". Dialogues in Clinical Neuroscience 13: 401–11.
- Norton, P.J.; Price, E.C. (2007). "A meta-analytic review of adult cognitive-behavioral treatment outcomes across the anxiety disorders". The Journal of Nervous and Mental Disease 195: 521–31. doi:10.1097/01.nmd.0000253843.70149.9a.
- Lilienfeld, O.S. (2007). "Psychological treatments that cause harm". Perspectives on Psychological Science 2: 53–70. doi:10.1111/j.1745-6916.2007.00029.x.
- Psychiatric Association, American (2002). "Ethical principles of psychologists and code of conduct". American Psychologist 57: 1067–73.
- Barlow, DH (2004). "Psychological treatments". American Psychologist 59: 869–78. doi:10.1037/0003-066x.59.9.869.
- Dimidjian, S; Hollon, DS (2010). "How would we know if psychotherapy were harmful?". American Psychologist 65: 34–49.
- Castonguay, GL; Boswell, FJ; Constantino, JM; Goldfried, RM; Hill, EC (2010). "Training implications of harmful effects of psychological treatments". American Psychologist 65: 34–49. doi:10.1037/a0017330.
- Bergin, AE (1971). The evaluation of therapeutic outcomes. Handbook of psychotherapy and behavior change, 217–70
- Neimeyer, RA (2000). "Searching for the meaning of meaning: grief therapy and the process of reconstruction". Death Studies 24: 541–58. doi:10.1080/07481180050121480.
- McNally, RJ; Bryant, RA; Ehlers, A (2003). "Does early psychological intervention promote recovery from posttraumatic stress?". Psychological Science in the Public Interest 4: 45–79. doi:10.1111/1529-1006.01421.
- Nishith, P; Resick, PA; Griffin, MG (2002). "Pattern of change in prolonged exposure and cognitive-processing therapy for female rape victims with posttraumatic stress disorder". Journal of Consulting and Clinical Psychology 70: 880–86. doi:10.1037/0022-006x.70.4.880.
- Gelso, CJ; Latts, MG; Gomez, MJ; Fassinger, RE (2002). "Countertransference management and therapy outcome: an initial evaluation". Journal of Clinical Psychology 58: 861–67. doi:10.1002/jclp.2010.
- Hill, CE; Mahalik, JR; Thompson, BJ (1989). "Therapist self-disclosure". Psychotherapy 26: 290–95. doi:10.1037/h0085438.
- Norcross, JC (2002), Psychotherapy relationships that work: therapist contributions and responsiveness to patients, New York: Oxford University Press.
- Hunsley, J; Di Giulio, G (2002), "Dodo bird, phoenix, or urban legend? The question of psychotherapy equivalence", The Scientific Review of Mental Health Practice 1: 11–22.
- Barlow, DH (2010), "The dodo bird– again– and again", The Behavior Therapist 33: 15–16.
- Ferguson, C; Brannick, M (2012), "Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses", Psychological Methods 17: 120–28, doi:10.1037/a0024445, PMID 21787082.
- Norcross, JC (1995), "Dispelling the Dodo bird verdict and the exclusivity myth in psychotherapy", Psychotherapy 32: 500–4, doi:10.1037/0033-3188.8.131.520.
- Reid, WJ (1997), "Evaluating the Dodo bird verdict: do all interventions have equivalent outcomes?", Social Work Research 21: 5–16, doi:10.1093/swr/21.1.5.
- Wilson, GT; Rachman, SJ (1983), "Meta-analysis and the evaluation of psychotherapy outcome: Limitations and liabilities", Journal of Consulting and Clinical Psychology 51: 54–64, doi:10.1037/0022-006x.51.1.54.
- Crits-Christoph (1997), "Limitations of the Dodo bird verdict and the role of clinical trials in psychotherapy research: comment on Wampold et al. (1997)", Psychological Bulletin 122: 216–20, doi:10.1037/0033-2909.122.3.216.
- Kazdin, AE; Bass, D (1989), "Power to detect differences between alternative treatments in comparative psychotherapy outcome research", Journal of Consulting and Clinical Psychology 57: 138–47, doi:10.1037/0022-006x.57.1.138.
- Norcross, JC; Rossi, JS (1994), "Looking weakly in all the wrong places? Comment of Shapiro et al. (1994)", Journal of Consulting and Clinical Psychology 62: 535–38, doi:10.1037/0022-006x.62.3.535.
- Chambless, Dianne (2002), "Beware the Dodo Bird: the dangers of overgeneralization", Clinical Psychology: Science and Practice (commentaries) 9: 13–16, doi:10.1093/clipsy/9.1.13.
- Beecham, J; Hallam, A; Knapp, M; Baines, B; Fenyo, A; Ashbury, M (1997), "Costing care in hospital and in the community", in Leff, J, Care in the community: illusion or reality?, pp. 93–108.
- Glasgow, RE; Klesges, LM; Dzewaltowski, DA; Bull, SS; Esta-brooks, P, "The future of health behavior change research: What is needed to improve translation of research into health promotion practice?", Annals of Behavioral Medicine 27: 3–12.
- Woolf, SH (2008), "The meaning of translational research and why it matters", Journal of the American Medical Association 299: 211–13, doi:10.1001/jama.2007.26, PMID 12633190.