Automation of surgical skill assessment using a three-stage machine learning algorithm

Abstract

Surgical skills are associated with clinical outcomes. To improve surgical skills and thereby reduce adverse outcomes, continuous surgical training and feedback is required. Currently, assessment of surgical skills is a manual and time-consuming process which is prone to subjective interpretation. This study aims to automate surgical skill assessment in laparoscopic cholecystectomy videos using machine learning algorithms. To address this, a three-stage machine learning method is proposed: first, a Convolutional Neural Network was trained to identify and localize surgical instruments. Second, motion features were extracted from the detected instrument localizations throughout time. Third, a linear regression model was trained based on the extracted motion features to predict surgical skills. This three-stage modeling approach achieved an accuracy of 87 ± 0.2% in distinguishing good versus poor surgical skill. While the technique cannot reliably quantify the degree of surgical skill yet it represents an important advance towards automation of surgical skill assessment.

Applications of Haptic Technology, Virtual Reality, and Artificial Intelligence in Medical Training During the COVID-19 Pandemic

“…Moreover, instrument detection in a video and drawing centroid based on the orientation and movement of the instruments can reflect the focus and ability to plan moves in a surgeon. Utilizing these centroids and calculating the radius, distance, and relative orientation can aid with the classification based on skill level Lavanchy et al (2021) .…”

Assessment of open surgery suturing skill: Simulator platform, force-based, and motion-based metrics

“…Acceleration of the hand and rotation of the wrist were found to distinguish expert surgeons from novices ( 18 , 28 ). In addition, hand and/or surgical tool motion obtained from external video using Artificial Intelligence (AI) were also examined to extract motion-based metrics ( 36 – 39 ). Total duration, path length, and number of movements were found to be important for distinguishing between attendings and medical students ( 37 ).…”

Quantification of Robotic Surgeries with Vision-Based Deep Learning

“…For example, Hung et al 13 used automated performance metrics (APMs) in order to predict a patient's post-operative length of stay within a hospital. Another line of research has instead focused on exclusively exploiting live surgical videos from endoscopic cameras to classify surgical activity 16 , gestures 17,18 , and skills 19 , among other tasks [20][21][22] . Most recently, attention-based neural networks such as Transformers 23 have been used to distinguish between distinct surgical steps within a procedure [24][25][26][27] .…”