Anatomy segmentation in laparoscopic surgery: comparison of machine learning and human expertise – an experimental study

Abstract

Background: Lack of anatomy recognition represents a clinically relevant risk in abdominal surgery. Machine learning (ML) methods can help identify visible patterns and risk structures, however, their practical value remains largely unclear. Materials and Methods: Based on a novel dataset of 13195 laparoscopic images with pixel-wise segmentations of eleven anatomical structures, we developed specialized segmentation models for each structure and combined models for all anatomical structures using two state-of-the-art model architectures (DeepLabv3 and SegFormer), and compared segmentation performance of algorithms to a cohort of 28 physicians, medical students, and medical laypersons using the example of pancreas segmentation. Results: Mean Intersection-over-Union for semantic segmentation of intraabdominal structures ranged from 0.28 to 0.83 and from 0.23 to 0.77 for the DeepLabv3-based structure-specific and combined models, and from 0.31 to 0.85 and from 0.26 to 0.67 for the SegFormer-based structure-specific and combined models, respectively. Both the structure-specific and the combined DeepLabv3-based models are capable of near-real-time operation, while the SegFormer-based models are not. All four models outperformed at least 26 out of 28 human participants in pancreas segmentation. Conclusions: These results demonstrate that ML methods have the potential to provide relevant assistance in anatomy recognition in minimally-invasive surgery in near-real-time. Future research should investigate the educational value and subsequent clinical impact of respective assistance systems.

Enhancing surgical planning for abdominal tumors in children through advanced 3D visualization techniques: a systematic review of future prospects

“…The concept of cybernetic surgery was first proposed in a robotic liver segmentectomy using augmented reality ( 44 ). Machine learning has been demonstrated to be relevant to human expertise for real-time anatomy segmentation during laparoscopy ( 45 ), particularly for nervous structures ( 46 ). All these supportive devices will enhance the level of assistance for the procedure.…”

One model to use them all: training a segmentation model with complementary datasets

“…For the stomach subset, additional masks are available, annotating six of the remaining ten classes (abdominal wall, colon, liver, pancreas, small intestine, spleen) visible in this subset, resulting in one multi-class subset. This work uses the proposed split [ 14 ] into training, validation, and test set. To be able to compare to the multi-class subset, only the binary subsets of the contained classes are used, with the exception of the spleen.…”