Deep learning-based recognition of key anatomical structures during robot-assisted minimally invasive esophagectomy

Abstract

Objective To develop a deep learning algorithm for anatomy recognition in thoracoscopic video frames from robot-assisted minimally invasive esophagectomy (RAMIE) procedures using deep learning. Background RAMIE is a complex operation with substantial perioperative morbidity and a considerable learning curve. Automatic anatomy recognition may improve surgical orientation and recognition of anatomical structures and might contribute to reducing morbidity or learning curves. Studies regarding anatomy recognition in complex surgical procedures are currently lacking. Methods Eighty-three videos of consecutive RAMIE procedures between 2018 and 2022 were retrospectively collected at University Medical Center Utrecht. A surgical PhD candidate and an expert surgeon annotated the azygos vein and vena cava, aorta, and right lung on 1050 thoracoscopic frames. 850 frames were used for training of a convolutional neural network (CNN) to segment the anatomical structures. The remaining 200 frames of the dataset were used for testing the CNN. The Dice and 95% Hausdorff distance (95HD) were calculated to assess algorithm accuracy. Results The median Dice of the algorithm was 0.79 (IQR = 0.20) for segmentation of the azygos vein and/or vena cava. A median Dice coefficient of 0.74 (IQR = 0.86) and 0.89 (IQR = 0.30) were obtained for segmentation of the aorta and lung, respectively. Inference time was 0.026 s (39 Hz). The prediction of the deep learning algorithm was compared with the expert surgeon annotations, showing an accuracy measured in median Dice of 0.70 (IQR = 0.19), 0.88 (IQR = 0.07), and 0.90 (0.10) for the vena cava and/or azygos vein, aorta, and lung, respectively. Conclusion This study shows that deep learning-based semantic segmentation has potential for anatomy recognition in RAMIE video frames. The inference time of the algorithm facilitated real-time anatomy recognition. Clinical applicability should be assessed in prospective clinical studies.

Does Thoracic Duct Ligation at the Time of Esophagectomy Impact Long-Term Survival? An Individual Patient Data Meta-Analysis

“…However, TDR seems associated with an increased risk of short-term adverse outcomes such as chylothorax, left recurrent nerve palsy, pulmonary complications, and hemodynamic instability that may be somewhat related to the concomitant azygos vein ligation/resection [60][61][62][63][64]. Nevertheless, TDR with the preservation of the azygos vein has been reported during robot-assisted thoracoscopic esophagectomy with no substantial effect on the total number of harvested lymph nodes [65][66][67]. Despite the higher number of retrieved lymph nodes, the real oncologic effect of TDR remains unsolved with some authors reporting even worse 5-year survival rates and more distant organ metastases [56].…”