Artificial Intelligence and Human Expertise in Cleft Lip and Palate Care: A Comparative Study of Accuracy, Readability, and Treatment Quality

Abstract

This study assesses the utility of artificial intelligence (AI), particularly ChatGPT models (GPT-3.5, GPT-4, and GPT-4o), in responding to inquiries compared with human professionals, focusing on applications in patient education and care standardisation. The authors conducted a comparative analysis of responses from AI and human experts—including a resident, a general plastic surgeon, and craniofacial surgeons—to 36 basic and 14 controversial questions related to cleft lip and palate. Evaluation criteria included response accuracy, readability (applying Flesch Reading Ease and Flesch–Kincaid Reading Grade Level), and the quality of treatment information (using DISCERN and PEMAT-P scores). AI demonstrated moderate accuracy (55.6%) on basic questions, although lower than that of experienced craniofacial surgeons. In controversial topics, AI displayed superior consistency and accuracy compared with residents. GPT-4o excelled in readability and understandability metrics. DISCERN scores indicated higher-quality treatment information from AI than from less experienced human respondents. However, AI struggled with actionable insights and nuanced clinical judgment, underscoring the critical role of experienced surgeons. Although AI, particularly GPT-4o, shows potential in enhancing patient education and fostering standardised communication in cleft care, it remains a supplementary tool that cannot replace the clinical expertise and decision-making capabilities of seasoned professionals. Integrating AI with human expertise could enhance the delivery of comprehensive cleft care.