Spatial aggregation of holistically-nested convolutional neural networks for automated pancreas localization and segmentation

Abstract

Accurate and automatic organ segmentation from 3D radiological scans is an important yet challenging problem for medical image analysis. Specifically, as a small, soft, and flexible abdominal organ, the pancreas demonstrates very high inter-patient anatomical variability in both its shape and volume. This inhibits traditional automated segmentation methods from achieving high accuracies, especially compared to the performance obtained for other organs, such as the liver, heart or kidneys. To fill this gap, we present an automated system from 3D computed tomography (CT) volumes that is based on a two-stage cascaded approach-pancreas localization and pancreas segmentation. For the first step, we localize the pancreas from the entire 3D CT scan, providing a reliable bounding box for the more refined segmentation step. We introduce a fully deep-learning approach, based on an efficient application of holistically-nested convolutional networks (HNNs) on the three orthogonal axial, sagittal, and coronal views. The resulting HNN per-pixel probability maps are then fused using pooling to reliably produce a 3D bounding box of the pancreas that maximizes the recall. We show that our introduced localizer compares favorably to both a conventional non-deep-learning method and a recent hybrid approach based on spatial aggregation of superpixels using random forest classification. The second, segmentation, phase operates within the computed bounding box and integrates semantic mid-level cues of deeply-learned organ interior and boundary maps, obtained by two additional and separate realizations of HNNs. By integrating these two mid-level cues, our method is capable of generating boundary-preserving pixel-wise class label maps that result in the final pancreas segmentation. Quantitative evaluation is performed on a publicly available dataset of 82 patient CT scans using 4-fold cross-validation (CV). We achieve a (mean ± std. dev.) Dice similarity coefficient (DSC) of 81.27 ± 6.27% in validation, which significantly outperforms both a previous state-of-the art method and a preliminary version of this work that report DSCs of 71.80 ± 10.70% and 78.01 ± 8.20%, respectively, using the same dataset.

Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks

“…As a natural extension of the well‐known 2D FCN proposed by Long et al ,. 3D‐FCNs have been successfully applied to semantic segmentation tasks in medical imaging, such as liver segmentation, brain tumor segmentation, and pancreas segmentation . As demonstrated in these studies, the skip connections designed in 3D‐FCNs or 3D‐UNets were very important to help recover the full spatial resolution at the network outputs, which is suitable for voxel‐wise segmentation tasks.…”

DRINet for Medical Image Segmentation

“…pancreas). Furthermore, deep learning approaches are much faster than conventional methods [4], [39], [40].…”

Liver Segmentation from Multimodal Images Using HED-Mask R-CNN

“…Deep supervision used in HED that accounts for low-level predictions resulting in better edge map, is one of the reasons to consider HED in our method. We thus chose HED that automatically learns rich and important hierarchical representations from MRI/CT images to resolve the challenging ambiguity in edge [6]. Consequently, HEDbased profound system models have been effectively utilized in medical image analysis for brain tumor segmentation [12], prostate segmentation [1], pancreas localization, and segmentation [6], retinal blood vessel segmentation [11].…”