Patient Health Questionnaire
|Patient Health Questionnaire|
The Patient Health Questionnaire (PHQ) is a multiple-choice self-report inventory copyrighted by Pfizer Inc, that is used as a screening and diagnostic tool for mental health disorders of depression, anxiety, alcohol, eating, and somatoform disorders. It is the self-report version of the Primary Care Evaluation of Mental Disorders (PRIME-MD), a diagnostic tool developed in the mid-1990s by Pfizer Inc. The length of the original assessment limited its feasibility; consequently, a shorter version, consisting of 11 multi-part questions - the Patient Health Questionnaire was developed and validated.
In addition to the PHQ, a nine-item version to assess symptoms of depression, a seven-item version to assess symptoms of anxiety (GAD-7), and a 15-item version to detect somatic symptoms (PHQ-15) have been developed and validated. The PHQ-9, GAD-7, and the PHQ-15 were combined to create the PHQ-somatic, anxiety, depressive symptoms (PHQ-SADS). Though less commonly used, there are also brief versions of the PHQ-9 and GAD-7 that may be useful as screening tools in some settings. In recent years, the PHQ-9 has been validated for use in adolescents, and a version for adolescents was also developed and validated (PHQ-A). Although these tests were originally designed as self-report inventories they can also be administered by trained health care practitioners.
The PHQ is available in over 20 languages, available on the PHQ website. Both the original Patient Health Questionnaire and later variants are public domain resources; no fees or permissions are required for using or copying the measures.:3,7–8 Additionally, the measures have been validated in a number of different populations internationally.
- 1 Versions
- 2 Reliability
- 3 Validity
- 4 Limitation
- 5 See also
- 6 References
- 7 External links
The original Patient Health Questionnaire contains five modules; these contain questions about depressive, anxiety, somatoform, alcohol, and eating disorders. Designed for use in the primary care setting, it lacks coverage for disorders seen in psychiatric settings. Some modules are used independently, and variants have been developed based on the original items.
|Patient Health Questionnaire 9 item|
|Synonyms||PHQ-9; PHQ Quick Depression Assessment; Resident mood interview|
9-item depression scale
The PHQ-9, a tool specific to depression, scores each of the 9 DSM-IV related criteria based on the mood module from the original PRIME-MD. The PHQ-9 is both sensitive and specific in its diagnoses, which has led to its prominence in the primary care setting. This tool is used in a variety of different contexts, including clinical settings across the United States as well as research studies.
One study which used the PHQ-9, examined if college student displays of depression symptoms on Facebook were representative of offline symptoms. Results demonstrated that those who displayed depression symptoms on Facebook scored higher on the PHQ-9, suggesting that those who display depression symptoms on Facebook are experiencing them offline.
|Patient Health Questionnaire 2 item|
|Patient Health Questionnaire 4 item|
2-item depression screener
The Patient Health Questionnaire 2 item (PHQ-2) is an ultra-brief screening instrument containing the first two questions from the PHQ-9.:3 Two screening questions to assess the presence of a depressed mood and a loss of interest or pleasure in routine activities, and a positive response to either question indicates further testing is required. This version of the PHQ has been shown to have good diagnostic sensitivity but poor specificity.
4-item depression and anxiety screener
The Patient Health Questionnaire 4 item (PHQ-4) combines the PHQ-2 with the Generalized Anxiety Disorder 2 (GAD-2), an ultra-brief anxiety screener containing the first two questions from the Generalized Anxiety Disorder 7 (GAD-7).:3
15-item somatic scale
|Patient Health Questionnaire 15 item|
|Synonyms||PHQ-15, PHQ Somatic Symptom Scale|
The Patient Health Questionnaire 15 item (PHQ-15) contains the PHQ's somatic symptom scale.:3 It is a well-validated measure, which asks whether symptoms are present and about their severity. A brief version, the Somatic Symptom Scale - 8 was derived from PHQ-15. The development of the PHQ-15 helped address three main problems in the assessment and diagnosis of somatoform disorders. Firstly, traditional methods of diagnosing somatoform disorders would only capture about 20% of true cases due to the number of symptoms required to meet a diagnosis. Secondly, in order to attain more reliable and valid data, asseessments need to address more current rather than previous symptoms. Thirdly, continuing to adhere to the “medically unexplained” requirement for symptoms makes it very difficult to make a diagnosis because it is extremely hard to ascertain if a symptom is or is not part of a larger medical condition (ex: chronic fatigue and depression).
|Patient Health Questionnaire - Somatic, Anxiety, and Depressive Symptoms|
Somatic, anxiety, and depressive symptoms
The Patient Health Questionnaire - Somatic, Anxiety, and Depressive Symptoms (PHQ-SADS) screens for somatic, anxiety, and depressive symptoms using PHQ-9, GAD-7, and PHQ-15, plus the panic symptoms question from the original PHQ.:3
7-item anxiety scale
The GAD-7 is a 7-item scale designed to assess symptoms of anxiety. Each item is scored on a 0-to-3 point scale (“not at all” to “nearly every day”). Cut points of 5, 10, and 15 correspond to mild, moderate, and severe anxiety.
8-item depression scale
The PHQ-8 is an eight-item scale developed specifically to screen for depression in American epidemiological populations.
The PHQ-A is a four module self-report to evaluate depression, anxiety, substance use and eating disorders in adolescent primary care patients.
|Measure||Criterion||Rating* (adequate, good, excellent, too good)||Explanation|
|PHQ||Norms||Excellent||Multiple convenience and random samples, as well as research studies in both clinical and nonclinical sample|
|Internal consistency (Cronbach’s alpha, split half, etc.)||Good||Cronbach’s alpha reported at .88 for measuring depression|
|Inter-rater reliability||Good||Kappas range from .64-.81 for depression. Kappa for anxiety is .83|
|Test-retest reliability (stability)||No published studies formally checking test-retest reliability|
|Repeatability||No published studies formally checking repeatability|
|PHQ-9||Norms||Excellent||Multiple convenience and random samples, as well as research studies in both clinical and nonclinical samples.|
|Internal consistency (Cronbach’s alpha, split half, etc.)||Good||Cronbach’s alphas range from .83 to .89|
|Inter-rater reliability||Good||One study in Nigerian university students with found ranges between .83 and .92|
|Test-retest reliability (stability||Adequate||Correlation between administrations done within 48 hours was .84.|
|Repeatability||Not published||No published studies formally checking repeatability.|
|GAD-7||Norms||Excellent||Multiple convenience and random samples, as well as research studies in both clinical and nonclinical samples|
|Internal consistency (Cronbach’s alpha, split half, etc.)||Good||Cronbach’s alpha reported at .92|
|Inter-rater reliability||Good||The interviewer vs. self-rated correlation ranges from .83 and .84|
|Test-retest reliability (stability||Good||Reported as .83|
|Repeatability||Not published||No published studies formally checking repeatability|
|PHQ-15||Norms||Excellent||Two large studies with convenience and random samples used. One research studies (N=906) in clinical sample and one research study (N=6000) in nonclinical sample.|
|Internal consistency (Cronbach’s alpha, split half, etc.)||Good||Cronbach’s alpha reported at .80|
|Inter-rater reliability||No published studies formally checking inter-rater reliability|
|Test-retest reliability (stability)||Good||Kappa = .60 when administration was done within 2 weeks of first test|
|Repeatability||No published studies formally checking repeatability.|
|Measure||Criterion||Rating* (adequate, good, excellent, too good*)||Explanation with references|
|PHQ||Content validity||Good||Covers DSM-IV criteria for major depressive disorder, panic disorder, other anxiety disorder, bulimia nervosa, other depressive disorder, probable alcohol abuse or dependence, and somatoform and binge eating disorders|
|Construct validity (e.g., predictive, concurrent, convergent, and discriminant validity)||Adequate||Construct validity has not been fully established, and more substantial evidence of convergent and discriminant validity would be helpful. Validity is consistent with PRIME-MD.|
|Discriminative validity||Excellent||AUCs range from .89 to .92 for detecting depression|
|Validity generalization||No published studies formally checking validity generalization.|
|Treatment sensitivity||No published studies formally checking treatment sensitivity.|
|Clinical utility||Good||The PHQ is free and can be completed independently by the patient; it assesses a wide array of mental health concerns.|
|PHQ-9||Content validity||Excellent||Covers the DSM-IV criteria for major depressive disorder.|
|Construct validity (e.g., predictive, concurrent, convergent, and discriminant validity)||Good||Higher PHQ-9 scores were correlated with greater self-reported disability days, clinic visits, health-care utilization, as well as difficulties in activities and relationships.|
|Discriminative validity||Too excellent||An average sensitivity of .77 and specificity of .94 (corresponding to an AUC .94) in primary care settings suggests good discriminative validity in populations that are generally not depressed, but it may not perform as well in clinical populations.|
|Validity generalization||Variable||A meta-analysis of 27 samples suggested that performance of the PHQ-9 is highly heterogeneous; pooled sensitivity is low and specificity is high.|
|Treatment sensitivity||Good||In a treatment study using three medical outpatient cohorts, the PHQ-9 has been shown to be sensitive to change over time
The PHQ-9 has been used in studies to effectively monitor change following cognitive behavioral treatment.
A meta analysis stated that the PHQ-9 had good treatment sensitivity.
|Clinical utility||Good||The PHQ-9 is brief, free to use, and easy to score. It has good specificity, but the poor sensitivity could lead to false negatives, which is a problem for a screening tool. It is likely to perform best in samples where the prevalence of depressive disorders is high. To improve clinical utility, meta-analyses suggest increasing cut score to 10 or higher to improve sensitivity.|
|GAD-7||Content validity||Good||Covers seven of the core symptoms for generalized anxiety disorder.|
|Construct validity (e.g., predictive, concurrent, convergent, and discriminant validity)||Good||Scores correlate with the Beck Anxiety Inventory (r= .72) and the anxiety subscale of the SCL-90 (r=.74).|
|Discriminative validity||Too excellent||AUC for detecting generalized anxiety disorder was .91, for panic disorder AUC= .85 for panic disorder, AUC=.83 for social anxiety disorder, and AUC=.83 for PTSD.|
|Validity generalization||Good||Validity has been established across multiple populations.|
|Treatment sensitivity||Good||The GAD-7 showed good sensitivity to treatment effects in two randomized-controlled trials.|
|Clinical utility||Excellent||The GAD-7 is brief, free to use, and easy to score. It is sensitive to change following treatment. There is some evidence that elderly people may require some help to complete the scale accurately.|
|PHQ-15||Content validity||Good||Scores correspond well to DSM-IV somatoform diagnoses from the SCID and General Health Questionnaire-15.|
|Construct validity (e.g., predictive, concurrent, convergent, and discriminant validity)||Adequate||PHQ-15 scores correlated with medically unexplained symptom counts (r=.52) measured via an independent psychiatric review and with the General Health Questionnaire-15.|
|Discriminative validity||Excellent||Sensitivity was 78% and specificity was 71% for a DSM-IV diagnosis of somatoform disorder, corresponding to an AUC of .76.|
|Validity generalization||Although the PHQ-15 is currently being used in major studies in several European countries and Australia. there is evidence that it does not perform as well in Hispanic populations.|
|Treatment sensitivity||Unknown||Meta-analysis states that the treatment sensitivity for the PHQ-15 has not been researched much, but there is some support that the PHQ-15 is sensitive to treatment.|
|Clinical utility||Good||The PHQ-15 is easy to use, free, and has a high discriminant and convergent validity, it has also been validated in many different clinical populations.|
The time period assessed by each scale could also be a limitation; the PHQ-9 asks about the last four weeks, whereas the GAD-7 focuses on the past two weeks, and the PHQ asks about various time periods from the last two weeks to the last six months. Depending on the time period in question, this may or may not require a revision (i.e., if you are interested in depression over the last six months, you might alter the instructions), which could impact the validity of the measure.
The scoring thresholds recommended are influenced by the samples in which they were validated and correspond with different levels of sensitivity and specificity, which may or may not match well with the intended use of the scale.
- Spitzer RL, Kroenke K, Williams JB, et al. (10 November 1999). "Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study". JAMA. 282 (18): 1737–44. PMID 10568646. doi: .
- Kroenke, Kurt; Spitzer, Robert L.; Williams, Janet B. W.; Löwe, Bernd (2010-07-01). "The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review". General Hospital Psychiatry. 32 (4): 345–359. ISSN 1873-7714. PMID 20633738. doi:10.1016/j.genhosppsych.2010.03.006.
- Kroenke, Kurt; Spitzer, Robert L; Williams, Janet B W (2017-05-31). "The PHQ-9". Journal of General Internal Medicine. 16 (9): 606–613. ISSN 0884-8734. PMC . PMID 11556941. doi:10.1046/j.1525-1497.2001.016009606.x.
- Kroenke, Kurt; Spitzer, Robert L.; Williams, Janet B. W.; Löwe, Bernd (2010-07-01). "The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review". General Hospital Psychiatry. 32 (4): 345–359. ISSN 0163-8343. doi:10.1016/j.genhosppsych.2010.03.006.
- Allgaier, Antje-Kathrin; Pietsch, Kathrin; Frühe, Barbara; Sigl-Glöckner, Johanna; Schulte-Körne, Gerd (2012-10-01). "Screening for Depression in Adolescents: Validity of the Patient Health Questionnaire in Pediatric Care". Depression and Anxiety. 29 (10): 906–913. ISSN 1520-6394. doi:10.1002/da.21971.
- Johnson, Jeffery; Harris; Spitzer; Williams (March 2002). "The patient health questionnaire for adolescents". Journal of Adolescent Health. 30: 196–204.
- Arroll B, Goodyear-Smith F, Crengle S, Gunn J, Kerse, Ngaire, Fishman T, Falloon K, Hatcher S (July 2010). "Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population". Annals of Family Medicine. 8 (4): 348–53. PMC . PMID 20644190. doi: .
- "Instructions for Patient Health Questionnaire (PHQ) and GAD-7 Measures" (PDF). Patient Health Questionnaire (PHQ) Screeners. Pfizer. 16 April 2014.
- Tsai, Fang-Ju; Huang, Yu-Hsin; Liu, Hui-Ching; Huang, Kuo-Yang; Huang, Yen-Hsun; Liu, Shen-Ing (2014-02-01). "Patient Health Questionnaire for School-Based Depression Screening Among Chinese Adolescents". Pediatrics. 133 (2): e402–e409. ISSN 0031-4005. PMID 24446447. doi:10.1542/peds.2013-0204.
- Chen, Teddy M.; Huang, Frederick Y.; Chang, Christine; Chung, Henry (2006-07-01). "Using the PHQ-9 for Depression Screening and Treatment Monitoring for Chinese Americans in Primary Care". Psychiatric Services. 57 (7): 976–981. ISSN 1075-2730. doi:10.1176/ps.2006.57.7.976.
- Adewuya, Abiodun O.; Ola, Bola A.; Afolabi, Olusegun O. (2006-11-01). "Validity of the patient health questionnaire (PHQ-9) as a screening tool for depression amongst Nigerian university students". Journal of Affective Disorders. 96 (1-2): 89–93. ISSN 0165-0327. PMID 16857265. doi:10.1016/j.jad.2006.05.021.
- Blacker D (2009). "Psychiatric Rating Scales". In Sadock BJ, Sadock VA, Ruiz P. Kaplan & Sadock's Comprehensive Textbook of Psychiatry (9th ed.). p. 1042. ISBN 978-0-7817-6899-3.
- Kroenke K, Spitzer RL, Williams JB (September 2001). "The PHQ-9: Validity of a brief depression severity measure". Journal of General Internal Medicine. 16 (9): 606–613. PMC . PMID 11556941. doi:10.1046/j.1525-1497.2001.016009606.x.
- Moreno MA, Christakis DA, Egan KG, Jelenchick LA, Cox E, Young H, Villiard H, Becker T (July 2012). "A pilot evaluation of associations between displayed depression references on Facebook and self-reported depression using a clinical scale". Journal of Behavioral Health Services & Research. 39 (3): 295–304. PMC . PMID 21863354. doi:10.1007/s11414-011-9258-7.
- Whooley MA, Avins AL, Miranda J, Browner WS (July 1997). "Case-finding instruments for depression: Two questions are as good as many". Journal of General Internal Medicine. 12 (7): 439–45. PMC . PMID 9229283. doi:10.1046/j.1525-1497.1997.00076.x.
- Kroenke, K; Spitzer, RL; Williams, JB (March 2002). "The PHQ-15: validity of a new measure for evaluating the severity of somatic symptoms". Psychosomatic Medicine. 64 (2): 258–66. PMID 11914441. (Subscription required (. ))
- Gierk, B; Kohlmann, S; Kroenke, K; Spangenberg, L; Zenger, M; Brähler, E; Löwe, B (March 2014). "The Somatic Symptom Scale–8 (SSS-8): A Brief Measure of Somatic Symptom Burden". JAMA Internal Medicine. 174 (3): 399–407. PMID 24276929. doi: .
- Kroenke, K; Spitzer, RL; Williams, JB; Löwe, B (July 2010). "The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: A systematic review". General Hospital Psychiatry. 32 (4): 345–59. PMID 20633738. doi:10.1016/j.genhosppsych.2010.03.006. (Subscription required (. ))
- Spitzer, Robert L.; Kroenke, Kurt; Williams, Janet B. W.; Löwe, Bernd (2006-05-22). "A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7". Archives of Internal Medicine. 166 (10): 1092–1097. ISSN 0003-9926. doi:10.1001/archinte.166.10.1092.
- Kroenke, Kurt; Strine, Tara W.; Spitzer, Robert L.; Williams, Janet B. W.; Berry, Joyce T.; Mokdad, Ali H. (2009-04-01). "The PHQ-8 as a measure of current depression in the general population". Journal of Affective Disorders. 114 (1-3): 163–173. ISSN 1573-2517. PMID 18752852. doi:10.1016/j.jad.2008.06.026.
- Löwe, Burnd; Spitzer; Gräfe; Kroenke; Quenter; Zipfel; Buchholz; Witte; Herzog (February 2004). "Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians’ diagnoses". Journal of Affective Disorders. 78: 131–140.
- Löwe, Bernd; Gräfe, Kerstin; Zipfel, Stephan; Witte, Steffen; Loerch, Bernd; Herzog, Wolfgang. "Diagnosing ICD-10 Depressive Episodes: Superior Criterion Validity of the Patient Health Questionnaire". Psychotherapy and Psychosomatics. 73 (6): 386–390. doi:10.1159/000080393.
- Persoons, Phillippe; Luyckx, Koen; Desloovere, Christian; Vandenberghe, Joris; Fischler, Benjamin (2003-09-01). "Anxiety and mood disorders in otorhinolaryngology outpatients presenting with dizziness: validation of the self-administered PRIME-MD Patient Health Questionnaire and epidemiology". General Hospital Psychiatry. 25 (5): 316–323. ISSN 0163-8343. doi:10.1016/s0163-8343(03)00072-0.
- Wittkampf, Karin A.; Naeije, Leonie; Schene, Aart H.; Huyser, Jochanan; van Weert, Henk C. (2007-09-01). "Diagnostic accuracy of the mood module of the Patient Health Questionnaire: a systematic review". General Hospital Psychiatry. 29 (5): 388–395. ISSN 0163-8343. PMID 17888804. doi:10.1016/j.genhosppsych.2007.06.004.
- Manea, Laura; Gilbody, Simon; McMillan, Dean (2015-01-01). "A diagnostic meta-analysis of the Patient Health Questionnaire-9 (PHQ-9) algorithm scoring method as a screen for depression". General Hospital Psychiatry. 37 (1): 67–75. ISSN 1873-7714. PMID 25439733. doi:10.1016/j.genhosppsych.2014.09.009.
- Cameron, Isobel M; Crawford, John R; Lawton, Kenneth; Reid, Ian C (2008-01-01). "Psychometric comparison of PHQ-9 and HADS for measuring depression severity in primary care". The British Journal of General Practice. 58 (546): 32–36. ISSN 0960-1643. PMC . PMID 18186994. doi:10.3399/bjgp08X263794.
- van Ravesteijn, Hiske; Wittkampf, Karin; Lucassen, Peter; van de Lisdonk, Eloy; van den Hoogen, Henk; van Weert, Henk; Huijser, Jochanan; Schene, Aart; van Weel, Chris (2009-05-01). "Detecting somatoform disorders in primary care with the PHQ-15". Annals of Family Medicine. 7 (3): 232–238. ISSN 1544-1717. PMC . PMID 19433840. doi:10.1370/afm.985.
- Schat, Aaron C. H.; Kelloway, E. Kevin; Desmarais, Serge. "The Physical Health Questionnaire (PHQ): Construct Validation of a Self-Report Scale of Somatic Symptoms.". Journal of Occupational Health Psychology. 10 (4): 363–381. doi:10.1037/1076-8922.214.171.1243.
- Löwe, Bernd; Kroenke, Kurt; Herzog, Wolfgang; Gräfe, Kerstin (2004-07-01). "Measuring depression outcome with a brief self-report instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9)". Journal of Affective Disorders. 81 (1): 61–66. ISSN 0165-0327. PMID 15183601. doi:10.1016/S0165-0327(03)00198-8.
- "http://dx.doi.org.elibrary.einstein.yu.edu/10.1016/j.genhosppsych.2015.05.009". External link in
- Spitzer, Robert L.; Kroenke, Kurt; Williams, Janet B. W.; Löwe, Bernd (2006-05-22). "A brief measure for assessing generalized anxiety disorder: the GAD-7". Archives of Internal Medicine. 166 (10): 1092–1097. ISSN 0003-9926. PMID 16717171. doi:10.1001/archinte.166.10.1092.
- García-Campayo, Javier; Zamorano, Enric; Ruiz, Miguel A.; Pardo, Antonio; Pérez-Páramo, María; López-Gómez, Vanessa; Freire, Olga; Rejas, Javier (2010). "Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool". Health and Quality of Life Outcomes. 8: 8. ISSN 1477-7525. PMC . PMID 20089179. doi:10.1186/1477-7525-8-8.
- Löwe, Bernd; Decker, Oliver; Müller, Stefanie; Brähler, Elmar; Schellberg, Dieter; Herzog, Wolfgang; Herzberg, Philipp Yorck (2008-03-01). "Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population". Medical Care. 46 (3): 266–274. ISSN 0025-7079. PMID 18388841. doi:10.1097/MLR.0b013e318160d093.
- Dear, Blake F.; Titov, Nickolai; Sunderland, Matthew; McMillan, Dean; Anderson, Tracy; Lorian, Carolyn; Robinson, Emma (2011-09-01). "Psychometric Comparison of the Generalized Anxiety Disorder Scale-7 and the Penn State Worry Questionnaire for Measuring Response during Treatment of Generalised Anxiety Disorder". Cognitive Behaviour Therapy. 40 (3): 216–227. ISSN 1650-6073. PMID 21770844. doi:10.1080/16506073.2011.582138.
- Han, Changsu; Pae, Chi-Un; Patkar, Ashwin A.; Masand, Prakash S.; Kim, Ki Woong; Joe, Sook-Haeng; Jung, In-Kwa. "Psychometric Properties of the Patient Health Questionnaire–15 (PHQ–15) for Measuring the Somatic Symptoms of Psychiatric Outpatients". Psychosomatics. 50 (6): 580–585. doi:10.1016/s0033-3182(09)70859-x.
- Interian, Alejandro; Allen, Lesley A.; Gara, Michael A.; Escobar, Javier I.; Díaz-Martínez, Angélica M. "Somatic Complaints in Primary Care: Further Examining the Validity of the Patient Health Questionnaire (PHQ-15)". Psychosomatics. 47 (5): 392–398. doi:10.1176/appi.psy.47.5.392.
- Hunt, Melissa; Auriemma, Joseph; Cashaw, Ashara C. A. (2003-02-01). "Self-Report Bias and Underreporting of Depression on the BDI-II". Journal of Personality Assessment. 80 (1): 26–30. ISSN 0022-3891. PMID 12584064. doi:10.1207/S15327752JPA8001_10.
- Ph.D, Dror Ben-Zeev; Young, Michael A.; Madsen, Joshua W. (2009-08-01). "Retrospective recall of affect in clinically depressed individuals and controls". Cognition and Emotion. 23 (5): 1021–1040. ISSN 0269-9931. doi:10.1080/02699930802607937.
- Rettew, David C.; Lynch, Alicia Doyle; Achenbach, Thomas M.; Dumenci, Levent; Ivanova, Masha Y. (2009-09-01). "Meta-analyses of agreement between diagnoses made from clinical evaluations and standardized diagnostic interviews". International Journal of Methods in Psychiatric Research. 18 (3): 169–184. ISSN 1557-0657. doi:10.1002/mpr.289.
- Youngstrom, Eric A. (2014-03-01). "A Primer on Receiver Operating Characteristic Analysis and Diagnostic Efficiency Statistics for Pediatric Psychology: We Are Ready to ROC". Journal of Pediatric Psychology. 39 (2): 204–221. ISSN 0146-8693. PMC . PMID 23965298. doi:10.1093/jpepsy/jst062.