Developing Effective Automated Feedback in Temporal Bone Surgery Simulation

Abstract

Objective We aim to test the effectiveness, accuracy, and usefulness of an automated feedback system in facilitating skill acquisition in virtual reality surgery. Study Design We evaluate the performance of the feedback system through a randomized controlled trial of 24 students allocated to feedback and nonfeedback groups. Setting The feedback system was based on the Melbourne University temporal bone surgery simulator. The study was conducted at the simulation laboratory of the Royal Victorian Eye and Ear Hospital, Melbourne. Subjects and Methods The study participants were medical students from the University of Melbourne, who were asked to perform virtual cortical mastoidectomy on the simulator. The extent to which the drilling behavior of the feedback and nonfeedback groups differed was used to evaluate the effectiveness of the system. Its accuracy was determined through a postexperiment observational assessment of recordings made during the experiment by an expert surgeon. Its usability was evaluated using students’ self-reports of their impressions of the system. Results A Friedman’s test showed that there was a significant improvement in the drilling performance of the feedback group, χ2(1) = 14.450, P < .001. The postexperiment assessment demonstrated that the system provided timely feedback (when trainee behavior was detected) 88.6% of the time and appropriate feedback (accurate advice) 84.2% of the time. Participants’ opinions about the usefulness of the system were highly positive. Conclusion The automated feedback system was observed to be effective in improving surgical technique, and the provided feedback was found to be accurate and useful.

Computer‐Aided Design and 3‐Dimensional Printing for Costal Cartilage Simulation of Airway Graft Carving

“…This may be due to the heterogeneity of the participant groups, difficulty of the simulation task, lack of prior exposure by participants and limited corrective instruction provided during or between attempts. Increased number of attempts in the simulation may be necessary before baseline performance expectations or gains in skill can be quantified 2,10 . No differentiation could be made between educational modality (video versus live demonstration), which may be attributed to inadequate exposures to the instructional materials and limitations of the educational environment.…”

Performance Assessment for Mastoidectomy: State of the Art Review

“…15,19,20,23,24,25 The VR systems employed in these studies include the Stanford, OSU, University of Melbourne, Visible Ear, and VOXEL-MAN simulators.…”

Region-Specific Automated Feedback in Temporal Bone Surgery Simulation

“…A previous evaluation of the feedback system with stagebased random forest classifiers [12] showed that timely feedback was provided 88.6% of the time, and out of these feedback messages, 84.2% were accurate. This evaluation was based on the automated feedback provided to participants during the study.…”

“…Wijewickrema et al [11] used these models to develop a feedback system that provides real-time advice on various aspects of surgical drilling technique. The system was shown to provide accurate and effective feedback [12].…”