Robust Resolution-Enhanced Prostate Segmentation in Magnetic Resonance and Ultrasound Images through Convolutional Neural Networks

Abstract

Prostate segmentations are required for an ever-increasing number of medical applications, such as image-based lesion detection, fusion-guided biopsy and focal therapies. However, obtaining accurate segmentations is laborious, requires expertise and, even then, the inter-observer variability remains high. In this paper, a robust, accurate and generalizable model for Magnetic Resonance (MR) and three-dimensional (3D) Ultrasound (US) prostate image segmentation is proposed. It uses a densenet-resnet-based Convolutional Neural Network (CNN) combined with techniques such as deep supervision, checkpoint ensembling and Neural Resolution Enhancement. The MR prostate segmentation model was trained with five challenging and heterogeneous MR prostate datasets (and two US datasets), with segmentations from many different experts with varying segmentation criteria. The model achieves a consistently strong performance in all datasets independently (mean Dice Similarity Coefficient -DSC- above 0.91 for all datasets except for one), outperforming the inter-expert variability significantly in MR (mean DSC of 0.9099 vs. 0.8794). When evaluated on the publicly available Promise12 challenge dataset, it attains a similar performance to the best entries. In summary, the model has the potential of having a significant impact on current prostate procedures, undercutting, and even eliminating, the need of manual segmentations through improvements in terms of robustness, generalizability and output resolution.

Deep learning for fully automatic detection, segmentation, and Gleason grade estimation of prostate cancer in multiparametric magnetic resonance images

“…Then, all sequences were combined into a single multi-channel image, in which any missing sequences were left blanks (value of 0), such as the three DCE channels in every ProstateX image, or the channel in every IVO image. The intensity was normalized by applying Equation 1 to every channel of an image I independently, as introduced in Pellicer-Valero et al 44 . Regarding objective (2), the procedure for homogenizing lesion representations between datasets is described in “ Pre-processing ” section, and a special data augmentation employed to alleviate the problem of missing sequences is presented in “ Model training and validation ” section.…”

“…Accordingly, a cascading system of two segmentation CNNs, similar to the one introduced by Zhu et al 52 , was developed for automatic CG and PZ segmentation. As it can be seen in Supplementary Figure 1, the first CNN -a published model 44 based on the U-Net 53 CNN architecture with dense 54 and residual 55 blocks-, takes a prostate T2 image as input and produces a prostate segmentation mask as output. Then, the second CNN takes both the T2 image and the prostate segmentation mask obtained in the previous step and generates a CG segmentation mask as output.…”

Deep Learning for fully automatic detection, segmentation, and Gleason Grade estimation of prostate cancer in multiparametric Magnetic Resonance Images

“…Then, all sequences were combined into a single multi-channel image, in which any missing sequences were left blanks (value of 0), such as the three DCE channels in every ProstateX image, or the K trans channel in every IVO image. The intensity was normalized by applying 1 to every channel of an image I independently, as introduced in Pellicer-Valero et al 46 .…”

“…Accordingly, a cascading system of two segmentation CNNs, similar to the one introduced by Zhu et al 74 , was developed for automatic CG and PZ segmentation. As it can be seen in Figure 3, the first CNN (which is a published model 46 based on the U-Net(Ronneberger, Fischer, and Brox 2015) CNN architecture with dense 23 and residual 20 blocks) takes a prostate T2 image as input and produces a prostate segmentation mask as output. Then, the second CNN takes both the T2 image and the prostate segmentation mask obtained in the previous step and generates a CG segmentation mask as output.…”

3D EAGAN: 3D edge-aware attention generative adversarial network for prostate segmentation in transrectal ultrasound images

“…Ronneberger et al ( 24 ) proposed an encoder-decoder-based U-Net architecture for medical semantic segmentation, which utilizes the skip connection to integrate low-level features extracted by the encoder into the decoder. Inspired by these novel architectures, a large number of DCNN-based prostate segmentation methods ( 25 - 29 ) were proposed.…”