Personalized Medicine Transformed: ChatGPT’s Contribution to Continuous Renal Replacement Therapy Alarm Management in Intensive Care Units
7
Issued Date
2024-03-01
Resource Type
eISSN
20754426
Scopus ID
2-s2.0-85189154441
Journal Title
Journal of Personalized Medicine
Volume
14
Issue
3
Rights Holder(s)
SCOPUS
Bibliographic Citation
Journal of Personalized Medicine Vol.14 No.3 (2024)
Suggested Citation
Sheikh M.S., Thongprayoon C., Qureshi F., Suppadungsuk S., Kashani K.B., Miao J., Craici I.M., Cheungpasitporn W. Personalized Medicine Transformed: ChatGPT’s Contribution to Continuous Renal Replacement Therapy Alarm Management in Intensive Care Units. Journal of Personalized Medicine Vol.14 No.3 (2024). doi:10.3390/jpm14030233 Retrieved from: https://repository.li.mahidol.ac.th/handle/123456789/97907
Title
Personalized Medicine Transformed: ChatGPT’s Contribution to Continuous Renal Replacement Therapy Alarm Management in Intensive Care Units
Author's Affiliation
Corresponding Author(s)
Other Contributor(s)
Abstract
The accurate interpretation of CRRT machine alarms is crucial in the intensive care setting. ChatGPT, with its advanced natural language processing capabilities, has emerged as a tool that is evolving and advancing in its ability to assist with healthcare information. This study is designed to evaluate the accuracy of the ChatGPT-3.5 and ChatGPT-4 models in addressing queries related to CRRT alarm troubleshooting. This study consisted of two rounds of ChatGPT-3.5 and ChatGPT-4 responses to address 50 CRRT machine alarm questions that were carefully selected by two nephrologists in intensive care. Accuracy was determined by comparing the model responses to predetermined answer keys provided by critical care nephrologists, and consistency was determined by comparing outcomes across the two rounds. The accuracy rate of ChatGPT-3.5 was 86% and 84%, while the accuracy rate of ChatGPT-4 was 90% and 94% in the first and second rounds, respectively. The agreement between the first and second rounds of ChatGPT-3.5 was 84% with a Kappa statistic of 0.78, while the agreement of ChatGPT-4 was 92% with a Kappa statistic of 0.88. Although ChatGPT-4 tended to provide more accurate and consistent responses than ChatGPT-3.5, there was no statistically significant difference between the accuracy and agreement rate between ChatGPT-3.5 and -4. ChatGPT-4 had higher accuracy and consistency but did not achieve statistical significance. While these findings are encouraging, there is still potential for further development to achieve even greater reliability. This advancement is essential for ensuring the highest-quality patient care and safety standards in managing CRRT machine-related issues.