Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool
Issued Date
2024-11-04
Resource Type
eISSN
25743805
Scopus ID
2-s2.0-85210463122
Pubmed ID
39576645
Journal Title
JAMA network open
Volume
7
Issue
11
Rights Holder(s)
SCOPUS
Bibliographic Citation
JAMA network open Vol.7 No.11 (2024) , e2429630
Suggested Citation
Levis B., Bhandari P.M., Neupane D., Fan S., Sun Y., He C., Wu Y., Krishnan A., Negeri Z., Imran M., Rice D.B., Riehm K.E., Azar M., Levis A.W., Boruff J., Cuijpers P., Gilbody S., Ioannidis J.P.A., Kloda L.A., Patten S.B., Ziegelstein R.C., Harel D., Takwoingi Y., Markham S., Alamri S.H., Amtmann D., Arroll B., Ayalon L., Baradaran H.R., Beraldi A., Bernstein C.N., Bhana A., Bombardier C.H., Buji R.I., Butterworth P., Carter G., Chagas M.H., Chan J.C.N., Chan L.F., Chibanda D., Clover K., Conway A., Conwell Y., Daray F.M., de Man-van Ginkel J.M., Fann J.R., Fischer F.H., Field S., Fisher J.R.W., Fung D.S.S., Gelaye B., Gholizadeh L., Goodyear-Smith F., Green E.P., Greeno C.G., Hall B.J., Hantsoo L., Härter M., Hides L., Hobfoll S.E., Honikman S., Hyphantis T., Inagaki M., Iglesias-Gonzalez M., Jeon H.J., Jetté N., Khamseh M.E., Kiely K.M., Kohrt B.A., Kwan Y., Lara M.A., Levin-Aspenson H.F., Liu S.I., Lotrakul M., Loureiro S.R., Löwe B., Luitel N.P., Lund C., Marrie R.A., Marsh L., Marx B.P., McGuire A., Mohd Sidik S., Munhoz T.N., Muramatsu K., Nakku J.E.M., Navarrete L., Osório F.L., Pence B.W., Persoons P., Petersen I., Picardi A., Pugh S.L., Quinn T.J., Rancans E., Rathod S.D., Reuter K., Rooney A.G., Santos I.S., Schram M.T. Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool. JAMA network open Vol.7 No.11 (2024) , e2429630. doi:10.1001/jamanetworkopen.2024.29630 Retrieved from: https://repository.li.mahidol.ac.th/handle/20.500.14594/102280
Title
Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool
Author(s)
Levis B.
Bhandari P.M.
Neupane D.
Fan S.
Sun Y.
He C.
Wu Y.
Krishnan A.
Negeri Z.
Imran M.
Rice D.B.
Riehm K.E.
Azar M.
Levis A.W.
Boruff J.
Cuijpers P.
Gilbody S.
Ioannidis J.P.A.
Kloda L.A.
Patten S.B.
Ziegelstein R.C.
Harel D.
Takwoingi Y.
Markham S.
Alamri S.H.
Amtmann D.
Arroll B.
Ayalon L.
Baradaran H.R.
Beraldi A.
Bernstein C.N.
Bhana A.
Bombardier C.H.
Buji R.I.
Butterworth P.
Carter G.
Chagas M.H.
Chan J.C.N.
Chan L.F.
Chibanda D.
Clover K.
Conway A.
Conwell Y.
Daray F.M.
de Man-van Ginkel J.M.
Fann J.R.
Fischer F.H.
Field S.
Fisher J.R.W.
Fung D.S.S.
Gelaye B.
Gholizadeh L.
Goodyear-Smith F.
Green E.P.
Greeno C.G.
Hall B.J.
Hantsoo L.
Härter M.
Hides L.
Hobfoll S.E.
Honikman S.
Hyphantis T.
Inagaki M.
Iglesias-Gonzalez M.
Jeon H.J.
Jetté N.
Khamseh M.E.
Kiely K.M.
Kohrt B.A.
Kwan Y.
Lara M.A.
Levin-Aspenson H.F.
Liu S.I.
Lotrakul M.
Loureiro S.R.
Löwe B.
Luitel N.P.
Lund C.
Marrie R.A.
Marsh L.
Marx B.P.
McGuire A.
Mohd Sidik S.
Munhoz T.N.
Muramatsu K.
Nakku J.E.M.
Navarrete L.
Osório F.L.
Pence B.W.
Persoons P.
Petersen I.
Picardi A.
Pugh S.L.
Quinn T.J.
Rancans E.
Rathod S.D.
Reuter K.
Rooney A.G.
Santos I.S.
Schram M.T.
Bhandari P.M.
Neupane D.
Fan S.
Sun Y.
He C.
Wu Y.
Krishnan A.
Negeri Z.
Imran M.
Rice D.B.
Riehm K.E.
Azar M.
Levis A.W.
Boruff J.
Cuijpers P.
Gilbody S.
Ioannidis J.P.A.
Kloda L.A.
Patten S.B.
Ziegelstein R.C.
Harel D.
Takwoingi Y.
Markham S.
Alamri S.H.
Amtmann D.
Arroll B.
Ayalon L.
Baradaran H.R.
Beraldi A.
Bernstein C.N.
Bhana A.
Bombardier C.H.
Buji R.I.
Butterworth P.
Carter G.
Chagas M.H.
Chan J.C.N.
Chan L.F.
Chibanda D.
Clover K.
Conway A.
Conwell Y.
Daray F.M.
de Man-van Ginkel J.M.
Fann J.R.
Fischer F.H.
Field S.
Fisher J.R.W.
Fung D.S.S.
Gelaye B.
Gholizadeh L.
Goodyear-Smith F.
Green E.P.
Greeno C.G.
Hall B.J.
Hantsoo L.
Härter M.
Hides L.
Hobfoll S.E.
Honikman S.
Hyphantis T.
Inagaki M.
Iglesias-Gonzalez M.
Jeon H.J.
Jetté N.
Khamseh M.E.
Kiely K.M.
Kohrt B.A.
Kwan Y.
Lara M.A.
Levin-Aspenson H.F.
Liu S.I.
Lotrakul M.
Loureiro S.R.
Löwe B.
Luitel N.P.
Lund C.
Marrie R.A.
Marsh L.
Marx B.P.
McGuire A.
Mohd Sidik S.
Munhoz T.N.
Muramatsu K.
Nakku J.E.M.
Navarrete L.
Osório F.L.
Pence B.W.
Persoons P.
Petersen I.
Picardi A.
Pugh S.L.
Quinn T.J.
Rancans E.
Rathod S.D.
Reuter K.
Rooney A.G.
Santos I.S.
Schram M.T.
Author's Affiliation
Faculty of Medicine
Faculty of Medicine and Health Sciences
Ramathibodi Hospital
School of Medicine
School of Medicine and Public Health
Centre for Global Mental Health
Berliner Institut für Gesundheitsforschung
Iran University of Medical Sciences, Endocrine Research Center
NYU Shanghai
Duke-NUS Medical School
Niigata Seiryo University
Riga Stradins University
University of Zimbabwe
University of South Florida Health
Stanford University School of Medicine
Harvard T.H. Chan School of Public Health
McMaster University
American College of Radiology, Reston
London School of Hygiene & Tropical Medicine
The University of Queensland
Faculty of Medicine
University of Washington School of Medicine
Hospital Universitari Germans Trias i Pujol
The University of Edinburgh
University of Rochester Medical Center
University of North Texas
University of Technology Sydney
Institut Lady Davis de Recherches Médicales
New York University
Samsung Medical Center, Sungkyunkwan university
College of Health Sciences
Technische Universität München
University of Birmingham
Monash University
University of Newcastle, College of Health, Medicine and Wellbeing
UNC Gillings School of Global Public Health
University of York
Leids Universitair Medisch Centrum
University of Pittsburgh
Universidade Federal de Pelotas
VA Medical Center
University of Waterloo
Stanford University
University of Wollongong
King's College London
Queensland University of Technology
Max Rady College of Medicine, University of Manitoba
Istituto Superiore Di Sanita
University of Manitoba
Singapore Institute of Mental Health
The Australian National University
Shimane University Faculty of Medicine
Vrije Universiteit Amsterdam
Universidade de São Paulo
Maastricht Universitair Medisch Centrum+
The University of Auckland
Duke University School of Medicine
Universitätsklinikum Hamburg-Eppendorf
The George Washington University
Chinese University of Hong Kong
Bar-Ilan University
Université McGill
Tan Tock Seng Hospital
Baylor College of Medicine
University of Calgary
Universiti Kebangsaan Malaysia
Instituto Nacional de Psiquiatría Ramon de la Fuente
College of Medical, Veterinary & Life Sciences
Universidad de Buenos Aires
Johns Hopkins University School of Medicine
Cumming School of Medicine
University of Cape Town
Healthcare Group Sint-Kamillus
Hospital Mesra Bukit Padang
Anxiety and Resilience Consultants
Group Practice for Psychotherapy and Psycho-oncology
Research Department
Butabika National Referral Hospital
Faculty of Medicine and Health Sciences
Ramathibodi Hospital
School of Medicine
School of Medicine and Public Health
Centre for Global Mental Health
Berliner Institut für Gesundheitsforschung
Iran University of Medical Sciences, Endocrine Research Center
NYU Shanghai
Duke-NUS Medical School
Niigata Seiryo University
Riga Stradins University
University of Zimbabwe
University of South Florida Health
Stanford University School of Medicine
Harvard T.H. Chan School of Public Health
McMaster University
American College of Radiology, Reston
London School of Hygiene & Tropical Medicine
The University of Queensland
Faculty of Medicine
University of Washington School of Medicine
Hospital Universitari Germans Trias i Pujol
The University of Edinburgh
University of Rochester Medical Center
University of North Texas
University of Technology Sydney
Institut Lady Davis de Recherches Médicales
New York University
Samsung Medical Center, Sungkyunkwan university
College of Health Sciences
Technische Universität München
University of Birmingham
Monash University
University of Newcastle, College of Health, Medicine and Wellbeing
UNC Gillings School of Global Public Health
University of York
Leids Universitair Medisch Centrum
University of Pittsburgh
Universidade Federal de Pelotas
VA Medical Center
University of Waterloo
Stanford University
University of Wollongong
King's College London
Queensland University of Technology
Max Rady College of Medicine, University of Manitoba
Istituto Superiore Di Sanita
University of Manitoba
Singapore Institute of Mental Health
The Australian National University
Shimane University Faculty of Medicine
Vrije Universiteit Amsterdam
Universidade de São Paulo
Maastricht Universitair Medisch Centrum+
The University of Auckland
Duke University School of Medicine
Universitätsklinikum Hamburg-Eppendorf
The George Washington University
Chinese University of Hong Kong
Bar-Ilan University
Université McGill
Tan Tock Seng Hospital
Baylor College of Medicine
University of Calgary
Universiti Kebangsaan Malaysia
Instituto Nacional de Psiquiatría Ramon de la Fuente
College of Medical, Veterinary & Life Sciences
Universidad de Buenos Aires
Johns Hopkins University School of Medicine
Cumming School of Medicine
University of Cape Town
Healthcare Group Sint-Kamillus
Hospital Mesra Bukit Padang
Anxiety and Resilience Consultants
Group Practice for Psychotherapy and Psycho-oncology
Research Department
Butabika National Referral Hospital
Corresponding Author(s)
Other Contributor(s)
Abstract
Importance: Test accuracy studies often use small datasets to simultaneously select an optimal cutoff score that maximizes test accuracy and generate accuracy estimates. Objective: To evaluate the degree to which using data-driven methods to simultaneously select an optimal Patient Health Questionnaire-9 (PHQ-9) cutoff score and estimate accuracy yields (1) optimal cutoff scores that differ from the population-level optimal cutoff score and (2) biased accuracy estimates. Design, Setting, and Participants: This study used cross-sectional data from an existing individual participant data meta-analysis (IPDMA) database on PHQ-9 screening accuracy to represent a hypothetical population. Studies in the IPDMA database compared participant PHQ-9 scores with a major depression classification. From the IPDMA population, 1000 studies of 100, 200, 500, and 1000 participants each were resampled. Main Outcomes and Measures: For the full IPDMA population and each simulated study, an optimal cutoff score was selected by maximizing the Youden index. Accuracy estimates for optimal cutoff scores in simulated studies were compared with accuracy in the full population. Results: The IPDMA database included 100 primary studies with 44 503 participants (4541 [10%] cases of major depression). The population-level optimal cutoff score was 8 or higher. Optimal cutoff scores in simulated studies ranged from 2 or higher to 21 or higher in samples of 100 participants and 5 or higher to 11 or higher in samples of 1000 participants. The percentage of simulated studies that identified the true optimal cutoff score of 8 or higher was 17% for samples of 100 participants and 33% for samples of 1000 participants. Compared with estimates for a cutoff score of 8 or higher in the population, sensitivity was overestimated by 6.4 (95% CI, 5.7-7.1) percentage points in samples of 100 participants, 4.9 (95% CI, 4.3-5.5) percentage points in samples of 200 participants, 2.2 (95% CI, 1.8-2.6) percentage points in samples of 500 participants, and 1.8 (95% CI, 1.5-2.1) percentage points in samples of 1000 participants. Specificity was within 1 percentage point across sample sizes. Conclusions and Relevance: This study of cross-sectional data found that optimal cutoff scores and accuracy estimates differed substantially from population values when data-driven methods were used to simultaneously identify an optimal cutoff score and estimate accuracy. Users of diagnostic accuracy evidence should evaluate studies of accuracy with caution and ensure that cutoff score recommendations are based on adequately powered research or well-conducted meta-analyses.