Generative multi-adversarial network for striking the right balance in abdominal image segmentation

Abstract

Purpose The identification of abnormalities that are relatively rare within otherwise normal anatomy is a major challenge for deep learning in the semantic segmentation of medical images. The small number of samples of the minority classes in the training data makes the learning of optimal classification challenging, while the more frequently occurring samples of the majority class hamper the generalization of the classification boundary between infrequently occurring target objects and classes. In this paper, we developed a novel generative multi-adversarial network, called Ensemble-GAN, for mitigating this class imbalance problem in the semantic segmentation of abdominal images. Method The Ensemble-GAN framework is composed of a single-generator and a multi-discriminator variant for handling the class imbalance problem to provide a better generalization than existing approaches. The ensemble model aggregates the estimates of multiple models by training from different initializations and losses from various subsets of the training data. The single generator network analyzes the input image as a condition to predict a corresponding semantic segmentation image by use of feedback from the ensemble of discriminator networks. To evaluate the framework, we trained our framework on two public datasets, with different imbalance ratios and imaging modalities: the Chaos 2019 and the LiTS 2017. Result In terms of the F1 score, the accuracies of the semantic segmentation of healthy spleen, liver, and left and right kidneys were 0.93, 0.96, 0.90 and 0.94, respectively. The overall F1 scores for simultaneous segmentation of the lesions and liver were 0.83 and 0.94, respectively. Conclusion The proposed Ensemble-GAN framework demonstrated outstanding performance in the semantic segmentation of medical images in comparison with other approaches on popular abdominal imaging benchmarks. The Ensemble-GAN has the potential to segment abdominal images more accurately than human experts.

Automatic quadriceps and patellae segmentation of MRI with cascaded U2‐Net and SASSNet deep learning model

“…CNN‐based methods, like the conventional baselines (e.g., U‐Net, 3D U‐Net, and U‐Net++), require a reasonably large training dataset to obtain high segmentation accuracy, 56 which can lead to memory issues with the high‐resolution images 37 . Moreover, imbalanced medical image data and high variability of target object shapes and locations often lead to unexpected segmentation results 57 . The anatomic symmetry and low distribution errors were ignored in previous models by a priori cropping 1,3,4,17–22,25,27 .…”

“…37 Moreover, imbalanced medical image data and high variability of target object shapes and locations often lead to unexpected segmentation results. 57 The anatomic symmetry and low distribution errors were ignored in previous models by a priori cropping. 1,3,4,[17][18][19][20][21][22]25,27 The second refinement stage alleviates the memory bottleneck by focusing only on the cropped high-resolution volume, provided by the low-resolution first stage.…”

Bayesian uncertainty estimation for detection of long-tailed and unseen conditions in medical images

“…17,18 At the algorithm level, conditional GANs are used with modification of the training loss 19,20 or ensemble learning. 21 Most of the GAN-based ensemble techniques modify the network architecture by training generative multi-discriminative networks, 22,23 multi-generative discriminative networks, 24 or a cascade of GANs. 25,26 For a comprehensive literature survey of GAN-related algorithms, we refer the reader to Ref.…”

State of the art: radiomics and radiomics-related artificial intelligence on the road to clinical translation

“… 14 Automatic segmentation, an active area of research, can save time in radiomics analyses which require large datasets with homogenous pre-processing. Some automatic segmentation methods in development include Hierarchical CNNs for breast tumours in DCE-MRI, 87 GAN-based segmentation for liver and brain tumours 88 and U-net-based methods. 89 , 90 U-net are of particular interest since they do not require as many training samples 91 as some of the other approaches, a limiting factor for DL in medical imaging.…”