An imputation study shows that missing outcome data can substantially bias pooled estimates in systematic reviews of patient-reported outcomes
Issued Date
2026-03-01
Resource Type
ISSN
08954356
eISSN
18785921
Scopus ID
2-s2.0-105028376467
Pubmed ID
41461360
Journal Title
Journal of Clinical Epidemiology
Volume
191
Rights Holder(s)
SCOPUS
Bibliographic Citation
Journal of Clinical Epidemiology Vol.191 (2026)
Suggested Citation
Shen Y., Li Z., Gu X., Yao Y., Parpia S., Heels-Ansdell D., Chang Y., Wang Y., Shi Q., Hao Q., Jadid S.M., Jiravichitchai T., Kuriyama A., Shang Z., Wang Y., Zhao Y., Gao Y., Du L., Huang J., Guyatt G. An imputation study shows that missing outcome data can substantially bias pooled estimates in systematic reviews of patient-reported outcomes. Journal of Clinical Epidemiology Vol.191 (2026). doi:10.1016/j.jclinepi.2025.112120 Retrieved from: https://repository.li.mahidol.ac.th/handle/123456789/114479
Title
An imputation study shows that missing outcome data can substantially bias pooled estimates in systematic reviews of patient-reported outcomes
Author's Affiliation
Shandong University
McMaster University
Rijksuniversiteit Groningen
West China School of Medicine/West China Hospital of Sichuan University
Tongji Medical College of Huazhong University of Science and Technology
Beijing University of Chinese Medicine
Chengdu University of Traditional Chinese Medicine
McMaster University, Faculty of Health Sciences
McMaster University, Faculty of Science
Faculty of Medicine Ramathibodi Hospital, Mahidol University
The Second Affiliated Hospital of Chongqing Medical University
MAGIC Evidence Ecosystem Foundation
McMaster University
Rijksuniversiteit Groningen
West China School of Medicine/West China Hospital of Sichuan University
Tongji Medical College of Huazhong University of Science and Technology
Beijing University of Chinese Medicine
Chengdu University of Traditional Chinese Medicine
McMaster University, Faculty of Health Sciences
McMaster University, Faculty of Science
Faculty of Medicine Ramathibodi Hospital, Mahidol University
The Second Affiliated Hospital of Chongqing Medical University
MAGIC Evidence Ecosystem Foundation
Corresponding Author(s)
Other Contributor(s)
Abstract
Background and Objectives Missing outcome data (hereafter referred to as “missing data,” typically due to loss to follow-up) is a major problem in randomized controlled trials (RCTs) and systematic reviews of RCTs. While prior work has examined the impact of missing binary outcomes, the influence of missing continuous patient-reported outcome measures (PROMs) on pooled effect estimates remains poorly understood. We therefore assessed the risk of bias introduced by missing data in systematic reviews of PROMs. Study Design and Setting We selected a representative sample of 100 systematic reviews that included meta-analyses reporting a statistically significant effect on a continuous patient-reported efficacy outcome. We applied four increasingly stringent imputation strategies based on the grading of recommendations assessment, development, and evaluation (GRADE) approach, along with three alternative approaches for handling studies in which investigators had already imputed results for missing data. We also conducted Firth logistic regression analyses to identify factors associated with crossing the null after imputation. Results Results from 100 systematic reviews that included 1298 RCTs proved similar across all three approaches to addressing imputed data. Using the least stringent strategy for imputing missing data, the percentage of meta-analyses in which the 95% CI crossed the null proved under 4%. Applying the next most stringent strategy, the percentage of CIs that crossed the null increased to 47.9%. Percentages crossing the null increased only marginally for the two most stringent approaches, crossing up to 53.1% in the next most stringent and 54.2% in the most stringent. Firth logistic regression identified two significant predictors of crossing the null after imputation: a higher average missing data (odds ratio [OR] 1.23, 95% CI: 1.11–1.43 per 1% increase in missing data) and a larger magnitude of the treatment effect, which was associated with lower odds of crossing the null (OR 0.70, 95% CI: 0.39–0.91 per 1 standardized mean difference increase). Neither database type (Cochrane vs. non-Cochrane) nor duration of follow-up proved associated with CI crossing the null. Conclusion A plausible imputation approach to test the potential risk of bias as a result of missing data in studies addressing treatment effects on PROMs resulted in 95% CIs in a high proportion of studies initially suggesting benefit crossing the null. The greater the proportion of missing data and the smaller the treatment effect, the more likely the CI crossed the null. Systematic review authors may consider formally testing the robustness of their results with respect to missing data. Plain Language Summary When studies included in a systematic review have missing outcome data, the study results may be biased and therefore misleading. If there is not much missing data, this is not a problem. If there is lots of missing data, it can be a big problem. Researchers have suggested ways of highlighting how much of a problem missing data represents. This study compared four methods for handling missing data in continuous outcomes that measures patients’ experience, we call these patient-reported outcomes. We found that reasonable approaches to assessing the possible bias from missing data in systematic reviews and meta-analyses frequently highlight substantial problems and suggest that more guarded conclusions may be warranted in systematic reviews. These findings highlight the importance of fully considering the amount of missing data when interpreting the results of systematic reviews.