Simultaneous Super-Resolution and Segmentation Using a Generative Adversarial Network: Application to Neonatal Brain MRI

Pham, Chi-Hieu; Tor-Díez, Carlos; Meunier, Hélène; Bednarek, Nathalie; Fablet, Ronan; Passat, Nicolas; Rousseau, François

doi:10.1109/isbi.2019.8759255

Cited by 11 publications

(48 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have further investigated CNNbased architectures for image SR. Among others, the following features have been reported to improve SR performance: an increased depth of the network (Kim et al 2016a), residual block (with batch normalization and skip connection) (Ledig et al 2017), sub-pixel layer (Shi et al 2016), perceptual loss function (instead of mean squared error-based cost functions) (Johnson et al 2016;Ledig et al 2017;Zhao et al 2017), recurrent networks (Kim et al 2016b), generative adversarial networks (Ledig et al 2017;Pham et al 2019). Very recently, Chen et al (2018) proposed a 3D version of densely connected networks (Huang et al 2017) for brain MRI SR.…”

Section: Introductionmentioning

confidence: 99%

Multiscale brain MRI super-resolution using deep 3D convolutional networks

Pham¹,

Tor-Díez²,

Meunier³

et al. 2019

Computerized Medical Imaging and Graphics

133

101

View full text Add to dashboard Cite

The purpose of super-resolution approaches is to overcome the hardware limitations and the clinical requirements of imaging procedures by reconstructing high-resolution images from low-resolution acquisitions using post-processing methods. Super-resolution techniques could have strong impacts on structural magnetic resonance imaging when focusing on cortical surface or fine-scale structure analysis for instance. In this paper, we study deep three-dimensional convolutional neural networks for the super-resolution of brain magnetic resonance imaging data. First, our work delves into the relevance of several factors in the performance of the purely convolutional neural network-based techniques for the monomodal super-resolution: optimization methods, weight initialization, network depth, residual learning, filter size in convolution layers, number of the filters, training patch size and number of training subjects. Second, our study also highlights that one single network can efficiently handle multiple arbitrary scaling factors based on a multiscale training approach. Third, we further extend our super-resolution networks to the multimodal super-resolution using intermodality priors. Fourth, we investigate the impact of transfer learning skills onto super-resolution performance in terms of generalization among different datasets. Lastly, the learnt models are used to enhance real clinical low-resolution images. Results tend to demonstrate the potential of deep neural networks with respect to practical medical image applications.

show abstract

Section: Introductionmentioning

confidence: 99%

Multiscale brain MRI super-resolution using deep 3D convolutional networks

Pham¹,

Tor-Díez²,

Meunier³

et al. 2019

Computerized Medical Imaging and Graphics

133

101

View full text Add to dashboard Cite

show abstract

“…Comparison With Supervised Super-Resolution Methods: Supervised deep-learning super-resolution methods developed by [41,57] were evaluated on a subset of 20 neonatal brain MRIs from the dHCP dataset. Table III lists results as reported by [41] together with quantitative evaluation of our proposed approach on the same dataset (see Section VI-B1). Although methods of [41,57] used high-resolution groundtruth data for model training their results can be used to put results reported in this work into perspective.…”

Section: ) Resultsmentioning

confidence: 99%

“…In the experiments, three publicly available MRI datasets were used: cardiac cine MRI from the MICCAI 2017 Automated Cardiac Diagnosis Challenge (ACDC) ( [38]); neonatal brain MRI of the developing Human Connectome Project (dHCP) ( [39]) and adult brain MRI from the OASIS project ( [40]). Evaluation on neonatal and adult brain MRI enabled performance comparison with related unsupervised [28,29] and supervised [41] super-resolution methods. Finally, to demonstrate that our model is invariant to MRI scanners and voxel intensity distributions, we apply a model trained on cardiac MRI scans from the ACDC dataset to cardiac MRIs of the Sunnybrook dataset [42].…”

mentioning

confidence: 99%

Autoencoding Low-Resolution MRI for Semantically Smooth Interpolation of Anisotropic MRI

Sander,

de Vos,

Išgum

2022

Preprint

View full text Add to dashboard Cite

High-resolution medical images are beneficial for analysis but their acquisition may not always be feasible. Alternatively, high-resolution images can be created from low-resolution acquisitions using conventional upsampling methods, but such methods cannot exploit high-level contextual information contained in the images. Recently, better performing deep-learning based super-resolution methods have been introduced. However, these methods are limited by their supervised character, i.e. they require high-resolution examples for training. Instead, we propose an unsupervised deep learning semantic interpolation approach that synthesizes new intermediate slices from encoded low-resolution examples. To achieve semantically smooth interpolation in through-plane direction, the method exploits the latent space generated by autoencoders. To generate new intermediate slices, latent space encodings of two spatially adjacent slices are combined using their convex combination. Subsequently, the combined encoding is decoded to an intermediate slice. To constrain the model, a notion of semantic similarity is defined for a given dataset. For this, a new loss is introduced that exploits the spatial relationship between slices of the same volume. During training, an existing in-between slice is generated using a convex combination of its neighboring slice encodings. The method was trained and evaluated using publicly available cardiac cine, neonatal brain and adult brain MRI scans. In all evaluations, the new method produces significantly better results in terms of Structural Similarity Index Measure and Peak Signal-to-Noise Ratio (p < 0.001 using one-sided Wilcoxon signed-rank test) than a cubic B-spline interpolation approach. Given the unsupervised nature of the method, high-resolution training data is not required and hence, the method can be readily applied in clinical settings.

show abstract

“…To this end, we propose a GAN-based approach, called SegSRGAN, which generates not only a perceptual super-resolved image, but also a segmentation map (cortical, in our application case) from a single lowresolution MR image. This article, which is an extended and improved version of the conference paper [34], is organized as fol-lows. In Section 2, we describe the formulation of the super-resolution and segmentation image problem we aim to tackle.…”

Section: Introductionmentioning

confidence: 99%

SegSRGAN: Super-resolution and segmentation using generative adversarial networks — Application to neonatal brain MRI

Delannoy

Pham

Cazorla

et al. 2020

Computers in Biology and Medicine

View full text Add to dashboard Cite

Background and objective: One of the main issues in the analysis of clinical neonatal brain MRI is the low anisotropic resolution of the data. In most MRI analysis pipelines, data are first re-sampled using interpolation or single image super-resolution techniques and then segmented using (semi-)automated approaches. In other words, image reconstruction and segmentation are then performed separately. In this article, we propose a methodology and a software solution for carrying out simultaneously high-resolution reconstruction and segmentation of brain MRI data. Methods: Our strategy mainly relies on generative adversarial networks. The network architecture is described in detail. We provide information about its implementation, focusing on the most crucial technical points (whereas complementary details are given in a dedicated GitHub repository). We illustrate the behaviour of the proposed method for cortex analysis from neonatal MR images. Results: The results of the method, evaluated quantitatively (Dice, peak signal-to-noise ratio, structural similarity, number of connected components) and qualitatively on a research dataset (dHCP) and a clinical one (Epirmex), emphasize the relevance of the approach, and its ability to take advantage of data-augmentation strategies. Conclusions: Results emphasize the potential of our proposed method / software with respect to practical medical applications. The method is provided as a freely available software tool, which allows one to carry out his/her own experiments, and involve the method for the super-resolution reconstruction and segmentation of arbitrary cerebral structures from any MR image dataset.

show abstract

Simultaneous Super-Resolution and Segmentation Using a Generative Adversarial Network: Application to Neonatal Brain MRI

Cited by 11 publications

References 11 publications

Multiscale brain MRI super-resolution using deep 3D convolutional networks

Multiscale brain MRI super-resolution using deep 3D convolutional networks

Autoencoding Low-Resolution MRI for Semantically Smooth Interpolation of Anisotropic MRI

SegSRGAN: Super-resolution and segmentation using generative adversarial networks — Application to neonatal brain MRI

Contact Info

Product

Resources

About