Uncertainty-aware approach for multiple imputation using conventional and machine learning models: a real-world data study
9
Issued Date
2025-12-01
Resource Type
eISSN
21961115
Scopus ID
2-s2.0-105003797373
Journal Title
Journal of Big Data
Volume
12
Issue
1
Rights Holder(s)
SCOPUS
Bibliographic Citation
Journal of Big Data Vol.12 No.1 (2025)
Suggested Citation
Wabina R.S., Looareesuwan P., Sonsilphong S., Teza H., Ponthongmak W., McKay G., Attia J., Pattanateepapon A., Panitchote A., Thakkinstian A. Uncertainty-aware approach for multiple imputation using conventional and machine learning models: a real-world data study. Journal of Big Data Vol.12 No.1 (2025). doi:10.1186/s40537-025-01136-3 Retrieved from: https://repository.li.mahidol.ac.th/handle/123456789/110022
Title
Uncertainty-aware approach for multiple imputation using conventional and machine learning models: a real-world data study
Corresponding Author(s)
Other Contributor(s)
Abstract
Missing data poses a significant challenge in clinical real-world studies, often arising from unplanned data collection, misplacement, patient loss to follow-up, and other factors. While multiple imputation by chained equations (MICE) is a widely used method, its sequential nature introduces uncertainty, potentially impacting the prediction model performance. We proposed and evaluated three uncertainty-aware functions (i.e., uncertainty sampling (US), probability of improvement (PI), and expected improvement (EI)) integrated with linear regression (LinearReg), decision tree (DT), random forest (RF), and extreme gradient boosting (XGBoost) using three large datasets: chronic kidney disease (CKD, n = 31,043), hypertension cohort from Ramathibodi Hospital (HT-RAMA, n = 140,047) and Khon Kaen University Hospital (HT-KKU, n = 108,942) with high missing rates. In the CKD cohort, uncertainty-aware models significantly improved performance (evaluated by root mean squared error (RMSE) and mean absolute error (MAE)) over standard MICE, except for XGBoost. LinearReg-EI performed best (RMSE 0.12, MAE 0.36), followed by RF-EI (RMSE 0.22, MAE 0.34), and DT-EI (RMSE 0.21, MAE 0.38). In HT-RAMA, LinearReg-US performed best (RMSE 0.24, MAE 8.15), outperforming RF-US (RMSE 0.92, MAE 8.58) and DT-PI (RMSE 0.96, MAE 8.74). Similarly, in HT-KKU, LinearReg-US performed best (RMSE 0.98, MAE 12.00), followed by RF-PI (RMSE 1.93, MAE 12.90) and DT-US (RMSE 2.10, MAE 12.63). Uncertainty-aware models produced imputed distributions closely resembling the original data, unlike standard MICE. Our findings suggest that incorporating uncertainty functions can improve MICE, particularly for LinearReg, RF and DT. Further research is warranted to validate these findings across diverse clinical settings and model types.