Intelligent Tutoring for Surgical Decision Making: a Planning-Based Approach

Abstract

Virtual reality simulation has had a significant impact on training of psychomotor surgical skills, yet there is still a lack of work on its use to teach surgical decision making. This is particularly noteworthy given the recognized importance of decision making in achieving positive surgical outcomes. With the objective of filling this gap, we have developed a system for teaching surgical decision making in the field of endodontics, by integrating a virtual reality simulation environment with a conversational intelligent tutor. This work presents SDMentor (Surgical Decision-making Mentor) – the first intelligent tutoring system for teaching surgical decision making. In this paper we focus on presenting the intelligent tutoring component of the training system. The design of the system and the teaching approaches are driven by information gained from an observational study of clinical teaching sessions. The tutoring system represents surgical actions and the procedure using a variant of the planning domain definition language (PDDL). Tutorial interaction places emphasis on teaching the rationale for decisions, as well as aspects of situation awareness. We evaluated the quality of the tutorial content by comparing it with tutorial feedback from ten experienced human tutors. We had three expert dental instructors rate the appropriateness of the tutorial feedback given by the human tutors as well as SDMentor over 20 scenarios. Bayesian analysis showed that tutoring interventions of SDMentor were significantly better than interventions by human tutors. To determine whether the expert scores may have been influenced by raters’ knowledge of whether interventions came from human tutors or SDMentor, we carried out a type of Turing test. Results show that the expert raters were able to correctly guess which interventions came from SDMentor only 15% of the time, compared to a random baseline accuracy of 9%.