Deep residual networks for automatic segmentation of laparoscopic videos of the liver

Abstract

Motivation: For primary and metastatic liver cancer patients undergoing liver resection, a laparoscopic approach can reduce recovery times and morbidity while offering equivalent curative results; however, only about 10% of tumours reside in anatomical locations that are currently accessible for laparoscopic resection. Augmenting laparoscopic video with registered vascular anatomical models from pre-procedure imaging could support using laparoscopy in a wider population. Segmentation of liver tissue on laparoscopic video supports the robust registration of anatomical liver models by filtering out false anatomical correspondences between pre-procedure and intra-procedure images. In this paper, we present a convolutional neural network (CNN) approach to liver segmentation in laparoscopic liver procedure videos. Method: We defined a CNN architecture comprising fully-convolutional deep residual networks with multi-resolution loss functions. The CNN was trained in a leave-one-patient-out cross-validation on 2050 video frames from 6 liver resections and 7 laparoscopic staging procedures, and evaluated using the Dice score. Results: The CNN yielded segmentations with Dice scores ≥0.95 for the majority of images; however, the inter-patient variability in median Dice score was substantial. Four failure modes were identified from low scoring segmentations: minimal visible liver tissue, inter-patient variability in liver appearance, automatic exposure correction, and pathological liver tissue that mimics non-liver tissue appearance. Conclusion: CNNs offer a feasible approach for accurately segmenting liver from other anatomy on laparoscopic video, but additional data or computational advances are necessary to address challenges due to the high inter-patient variability in liver appearance.

More Unlabelled Data or Label More Data? A Study on Semi-supervised Laparoscopic Image Segmentation

“…Both the medians of Dice score and Hausdorff distance from MT were significantly better (both p-values < 0.001). The median Dice scores on 13 folds ranged from 0.87 to 0.98, with a median of 0.97, therefore surpassed the previous study [6] (p-value = 0.008). Examples are shown in Fig.…”

“…Baseline Supervised Network (SL) The median Dice scores on 13 folds from the baseline supervised network trained using all labelled images ranged from 0.85 to 0.98 with a median of 0.95, compared with 0.78, 0.98 and 0.97 from the previous study [6], respectively. The difference was probably due to the change of loss function and the adoption of the U-net variant.…”

“…The weight of L 2 loss was fixed to 10 −5 throughout the experiments. The network output has the same size as the re-sampled input image, larger than 81 × 21 used in previous work [6]. In the loss used in the mean teacher training, λ = 0.1β with β increasing progressively, i.e.…”

Generating Large Labeled Data Sets for Laparoscopic Image Processing Tasks Using Unpaired Image-to-Image Translation

“…When the network was tested on the 13 staging procedures [4] containing data that had not been seen at all during training, the mean dice score using only real training data B v was 0.25, and improved to 0.77 when the network was pre-trained with the synthetic data B syn .…”

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

“…The community is however putting large efforts in this direction, as illustrated by the recent generation of the CaDIS dataset [124], which contains pixellevel annotations for 36 semantic classes in cataract surgery videos. Progress has also been achieved in specific areas, such as liver segmentation [125], lesion detection and characterisation during gastroscopy [126] or polyp detection during colonoscopy [17], [127]. Here again, deep learning is the stateof-the art, as demonstrated for polyp detection in a challenge organized at MICCAI 2015 [128].…”