Pattern-based real-time feedback for a temporal bone simulator

Abstract

Delivering automated real-time performance feedback in simulated surgical environments is an important and challenging task. We propose a framework based on patterns to evaluate surgical performance and provide feedback during simulated ear (temporal bone) surgery in a 3D virtual environment. Temporal bone surgery is composed of a number of stages with distinct aims and surgical techniques. To provide context-appropriate feedback we must be able to identify each stage, recognise when feedback is to be provided, and determine the nature of that feedback. To achieve these aims, we train pattern-based models using data recorded by a temporal bone simulator. We create one model to predict the current stage of the procedure and separate stage-specific models to provide humanfriendly feedback within each stage. We use 27 temporal bone simulation runs conducted by 7 expert ear surgeons and 6 trainees to train and evaluate our models. The results of our evaluation show that the proposed system identifies the stage of the procedure correctly and provides constructive feedback to assist surgical trainees in improving their technique.

Optimal Sub-Sequence Matching for the Automatic Prediction of Surgical Tasks

“…We have mentioned in the introduction how this information can help ensuring a smooth running of the surgery. Another application concerns the training of junior surgeons, where our system could be integrated in a simulation environment in order to provide help and feedback to the junior surgeon [16]. Our system could, on demand, provide a warning to the surgeon about his or her deviation from the standard practice of his or her colleagues.…”

Region-Specific Automated Feedback in Temporal Bone Surgery Simulation

“…Supervised learning techniques were used to classify expert and trainee behaviour based on the above stroke properties. In previous work, Zhou et al developed and validated two such classification models (based on random forests and emerging patterns) [9], [10]. These models were trained using pre-recorded expert and trainee performances on the simulator.…”

“…To achieve this, an expert otologist divided the procedure into sequential stages characterised by similar drilling technique. A separate classification model was trained for each stage, along with a classifier to detect the stage and choose the corresponding feedback model [10].…”

“…However, these types of feedback do not adequately consider the complex and multi-faceted nature of surgical expertise. Previous work suggests that there exist complex relationships between the various metrics that characterise surgical technique [8], [9], [10]. For example, some types of surgical technique may only be detected if we examine how two or more metrics vary together.…”

“…Zhou et al [9], [10] attempted to bridge this gap by dividing temporal bone surgery into a set of distinct stages and developing multi-variate machine learning models that define expert and trainee technique in each stage. Wijewickrema et al [11] used these models to develop a feedback system that provides real-time advice on various aspects of surgical drilling technique.…”