OPG-SPELL: Visual and Textual Explanations that Influence Understanding of the AI Model for Sex Prediction from Orthopantomograms

Abstract

As AI models become increasingly powerful, the field of Explainable AI (XAI) has gained significant attention. However, the effectiveness of explanations often varies depending on the audience's domain knowledge. This paper introduces OPG-SPELL (understanding of OrthoPantomoGram's artificial intelligence prediction through ShaPlEy value and LLms text generation), an approach combining visual and textual explanations to enhance understanding and trust in AI models. OPG-SPELL focuses on an AI model trained to classify sex based on orthopantomograms (panoramic radiographs of human jaw structure). Our method integrates visual explanations using the OPG-SHAP technique with textual explanations generated through Large Language Models' Retrieval-Augmented Generation. This dual-modality approach aims to bridge the gap between expert and non-expert users, making AI decisions more interpretable across diverse audiences. We conducted a user study with 24 participants to evaluate the influence of both visual and textual explanations on understanding, trust, and willingness to use the XAI tool. Our findings demonstrate the potential of OPG-SPELL to enhance user comprehension and confidence in AI-driven sex prediction from orthopantomograms, contributing to the broader field of intelligent user interfaces and Explainable AI.