Fully automatic, multiorgan segmentation in normal whole body magnetic resonance imaging (MRI), using classification forests (CFs), convolutional neural networks (CNNs), and a multi‐atlas (MA) approach

Abstract

Purpose: As part of a program to implement automatic lesion detection methods for whole body magnetic resonance imaging (MRI) in oncology, we have developed, evaluated, and compared three algorithms for fully automatic, multiorgan segmentation in healthy volunteers. Methods: The first algorithm is based on classification forests (CFs), the second is based on 3D convolutional neural networks (CNNs) and the third algorithm is based on a multi-atlas (MA) approach. We examined data from 51 healthy volunteers, scanned prospectively with a standardized, multiparametric whole body MRI protocol at 1.5 T. The study was approved by the local ethics committee and written consent was obtained from the participants. MRI data were used as input data to the algorithms, while training was based on manual annotation of the anatomies of interest by clinical MRI experts. Fivefold cross-validation experiments were run on 34 artifact-free subjects. We report three overlap and three surface distance metrics to evaluate the agreement between the automatic and manual segmentations, namely the dice similarity coefficient (DSC), recall (RE), precision (PR), average surface distance (ASD), root-mean-square surface distance (RMSSD), and Hausdorff distance (HD). Analysis of variances was used to compare pooled label metrics between the three algorithms and the DSC on a 'per-organ' basis. A Mann-Whitney U test was used to compare the pooled metrics between CFs and CNNs and the DSC on a 'per-organ' basis, when using different imaging combinations as input for training. Results: All three algorithms resulted in robust segmenters that were effectively trained using a relatively small number of datasets, an important consideration in the clinical setting. Mean overlap metrics for all the segmented structures were: CFs: DSC = 0.70 AE 0.18, RE = 0.73 AE 0.18, PR = 0.71 AE 0.14, CNNs: DSC = 0.81 AE 0.13, RE = 0.83 AE 0.14, PR = 0.82 AE 0.10, MA: DSC = 0.71 AE 0.22, RE = 0.70 AE 0.34, PR = 0.77 AE 0.15. Mean surface distance metrics for all the segmented structures were: CFs: ASD = 13.5 AE 11.3 mm, RMSSD = 34.6 AE 37.6 mm and HD = 185.7 AE 194.0 mm, CNNs; ASD = 5.48 AE 4.84 mm, RMSSD = 17.0 AE 13.3 mm and HD = 199.0 AE 101.2 mm, MA: ASD = 4.22 AE 2.42 mm, RMSSD = 6.13 AE 2.55 mm, and HD = 38.9 AE 28.9 mm. The pooled performance of CFs improved when all imaging combinations (T2w + T1w + DWI) were used as input, while the performance of CNNs deteriorated, but in neither case, significantly. CNNs with T2w images as input, performed significantly better than CFs with all imaging combinations as input for all anatomical labels, except for the bladder. Conclusions: Three state-of-the-art algorithms were developed and used to automatically segment major organs and bones in whole body MRI; good agreement to manual segmentations performed by clinical MRI experts was observed. CNNs perform favorably, when using T2w volumes as input. Using multimodal MRI data as input to CNNs did not improve the segmentation performance.

Automatic Multiorgan Segmentation via Multiscale Registration and Graph Cut

“…With the emergence of deep learning, neural networks have regained ground in medical image analysis [3]. Deep neural networks of fully convolutional [31]- [33] and especially of the U-net variant [34], [35] have achieved very good results in multiorgan segmentation. The effectiveness of these methods depends on the availability of large annotated datasets and GPUbased computing to optimize empirically defined network architectures that can have millions of parameters [35].…”

Small Organ Segmentation in Whole-Body MRI Using a Two-Stage FCN and Weighting Schemes

“…Therefore, we need to induce spatial priors in order to guide the second stage network with a better ROI. A multi-atlas technique was chosen for producing spatial priors, because with multi-atlas segmentation, although the segmentation accuracy is lower than CNN-based accuracy, it mostly generates good organ localisation [7]. As shown in Fig.1, the bladder segmentation result of multi-atlas is focused in one specific location instead of having multiple predictions scattered in other areas (shown in FCN probability maps).…”

Convolutional neural network‐based approach for segmentation of left ventricle myocardial scar from 3D late gadolinium enhancement MR images

“…Our approach is substantially different from this previously published work as it includes the use of a 3D CNN approach, a substantially larger dataset and provides a comparison to alternative methods. Previous studies by other investigators have shown that CNN‐based methods outperform conventional image analysis techniques for segmentation problems across a variety of organs, such as brain tumours, other brain structures, liver lesions, pulmonary nodules, and multiorgan segmentation applied to the normal whole body (MRI) . We, therefore, anticipate expansion of this paradigm across a variety of cardiovascular applications.…”