No abstract

Surgical skills can be improved by continuous surgical training and feedback, thus reducing adverse outcomes while performing an intervention. With the advent of new technologies, researchers now have the tools to analyze surgical instrument motion to differentiate surgeons’ levels of technical skill. Surgical skills assessment is time-consuming and prone to subjective interpretation. The surgical instrument detection and tracking algorithm analyzes the image captured by the surgical robotic endoscope and extracts the movement and orientation information of a surgical instrument to provide surgical navigation. This information can be used to label raw surgical video datasets that are used to form an action space for surgical skill analysis. Instrument detection and tracking is a challenging problem in MIS, including robot-assisted surgeries, but vision-based approaches provide promising solutions with minimal hardware integration requirements. This study offers an overview of the developments of assessment systems for surgical intervention analysis. The purpose of this study is to identify the research gap and make a leap in developing technology to automate the incorporation of new surgical skills. A prime factor in automating the learning is to create datasets with minimal manual intervention from raw surgical videos. This review encapsulates the current trends in artificial intelligence (AI) based visual detection and tracking technologies for surgical instruments and their application for surgical skill assessment.

No abstract

Purpose Image segmentation of instruments in the raw surgical videos is a critical component of intraoperative assistance softwares. Challenges include addressing rendered overlays occluding the instrument while providing pivotal input to instrument tracking frameworks and, train the segmentation process with limited labelled data available from surgical videos. Method The proposed adversarial network, InstruSegNet uses unpaired training (eliminating need for massive paired data) for automated multi‐class surgical instrument segmentation in raw surgical videos with complex backgrounds. The proposed method is applied for single/multiple robotic and rigid instruments and optimised on least square Generative Adversarial Networks loss. Result Promising validation has been conducted on the publicly available dataset. Proposed approach for multi‐class segmentation of robotic and rigid instruments meets outstanding performance in terms of accuracy and surpasses the existing methods. Application This work facilitates segmenting instrument information without manual interventions from raw videos providing means to code surgeon's actions for developing intelligent assistance software.