Generative Adversarial Networks to Synthesize Missing T1 and FLAIR MRI Sequences for Use in a Multisequence Brain Tumor Segmentation Model

Conte, Gian Marco; Weston, Alexander D.; Vogelsang, David C; Philbrick, Kenneth A.; Cai, Jason; Barbera, Maurizio; Sanvito, Francesco; Lachance, Daniel H.; Jenkins, Robert B.; Tobin, W. Oliver; Eckel‐Passow, Jeanette E.; Erickson, Bradley J.

doi:10.1148/radiol.2021203786

Cited by 74 publications

(57 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 41 MR image synthesis could also be applied to improve the brain tumor structures contouring in radiotherapy treatment planning by generating the missing MRI sequence information. 27 Radiotherapy treatment of brain cancer patients often requires more than one MR sequence for delineating the structures. Studies 19 , 27 have reported that brain lesions could be delineated on the synthesized MR images with reasonable accuracy; dice similarity coefficient between the synthesized and real brain MR images ranged from 0.71 to 0.97 for different sequences.…”

Section: Discussionmentioning

confidence: 99%

Deep learning‐based convolutional neural network for intramodality brain MRI synthesis

Osman

Tamam

2022

J Applied Clin Med Phys

View full text Add to dashboard Cite

Purpose The existence of multicontrast magnetic resonance (MR) images increases the level of clinical information available for the diagnosis and treatment of brain cancer patients. However, acquiring the complete set of multicontrast MR images is not always practically feasible. In this study, we developed a state‐of‐the‐art deep learning convolutional neural network (CNN) for image‐to‐image translation across three standards MRI contrasts for the brain. Methods BRATS’2018 MRI dataset of 477 patients clinically diagnosed with glioma brain cancer was used in this study, with each patient having T1‐weighted (T1), T2‐weighted (T2), and FLAIR contrasts. It was randomly split into 64%, 16%, and 20% as training, validation, and test set, respectively. We developed a U‐Net model to learn the nonlinear mapping of a source image contrast to a target image contrast across three MRI contrasts. The model was trained and validated with 2D paired MR images using a mean‐squared error (MSE) cost function, Adam optimizer with 0.001 learning rate, and 120 epochs with a batch size of 32. The generated synthetic‐MR images were evaluated against the ground‐truth images by computing the MSE, mean absolute error (MAE), peak signal‐to‐noise ratio (PSNR), and structural similarity index (SSIM). Results The generated synthetic‐MR images with our model were nearly indistinguishable from the real images on the testing dataset for all translations, except synthetic FLAIR images had slightly lower quality and exhibited loss of details. The range of average PSNR, MSE, MAE, and SSIM values over the six translations were 29.44–33.25 dB, 0.0005–0.0012, 0.0086–0.0149, and 0.932–0.946, respectively. Our results were as good as the best‐reported results by other deep learning models on BRATS datasets. Conclusions Our U‐Net model exhibited that it can accurately perform image‐to‐image translation across brain MRI contrasts. It could hold great promise for clinical use for improved clinical decision‐making and better diagnosis of brain cancer patients due to the availability of multicontrast MRIs. This approach may be clinically relevant and setting a significant step to efficiently fill a gap of absent MR sequences without additional scanning.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep learning‐based convolutional neural network for intramodality brain MRI synthesis

Osman

Tamam

2022

J Applied Clin Med Phys

View full text Add to dashboard Cite

show abstract

“…CycleGANs are a relatively novel type of conditional generative adversarial networks, which have received considerable attention because of their ability to capture the characteristics of a single image collection and to generate synthetic images in the absence of any paired training examples 16 , 32 . Previous generative adversarial network studies on brain imaging have mainly focused on generating missing brain MRI data 33 – 35 or creating high-resolution images from low-resolution images 36 , which requires ground truth sequences. However, no “ground truth” dataset of paired training examples (consisting of internal and external MRI examinations of identical patients at the same period) exists in real-world clinical practice.…”

Section: Discussionmentioning

confidence: 99%

Cycle-consistent adversarial networks improves generalizability of radiomics model in grading meningiomas on external validation

Park

Shin

Eom

et al. 2022

Sci Rep

View full text Add to dashboard Cite

The heterogeneity of MRI is one of the major reasons for decreased performance of a radiomics model on external validation, limiting the model’s generalizability and clinical application. We aimed to establish a generalizable radiomics model to predict meningioma grade on external validation through leveraging Cycle-Consistent Adversarial Networks (CycleGAN). In this retrospective study, 257 patients with meningioma were included in the institutional training set. Radiomic features (n = 214) were extracted from T2-weighted (T2) and contrast-enhanced T1 (T1C) images. After radiomics feature selection, extreme gradient boosting classifiers were developed. The models were validated in the external validation set consisting of 61 patients with meningiomas. To reduce the gap in generalization associated with the inter-institutional heterogeneity of MRI, the smaller image set style of the external validation was translated into the larger image set style of the institutional training set using CycleGAN. On external validation before CycleGAN application, the performance of the combined T2 and T1C models showed an area under the curve (AUC), accuracy, and F1 score of 0.77 (95% confidence interval 0.63–0.91), 70.7%, and 0.54, respectively. After applying CycleGAN, the performance of the combined T2 and T1C models increased, with an AUC, accuracy, and F1 score of 0.83 (95% confidence interval 0.70–0.97), 73.2%, and 0.59, respectively. Quantitative metrics (by Fréchet Inception Distance) showed that CycleGAN can decrease inter-institutional image heterogeneity while preserving predictive information. In conclusion, leveraging CycleGAN may be helpful to increase the generalizability of a radiomics model in differentiating meningioma grade on external validation.

show abstract

“…Accurate segmentation of brain tumors from MRI images is of enormous potential value for improved diagnosis ( 28 ). It can be done automatically to cope with the time-consuming disadvantage of manual segmentation ( 29 , 30 ). Considering that the MRI images in our study were also used in the BraTS challenge and that the ground truth of tumor segmentation for patients in our cohort are not all available, we used the pre-trained model on the BraTS challenge 2019 to delineate the regions of tumor lesions, which achieved the accuracy of 90.45% on the validation set ( 31 ).…”

Section: Methodsmentioning

confidence: 99%

Multimodal MRI Image Decision Fusion-Based Network for Glioma Classification

et al. 2022

View full text Add to dashboard Cite

PurposeGlioma is the most common primary brain tumor, with varying degrees of aggressiveness and prognosis. Accurate glioma classification is very important for treatment planning and prognosis prediction. The main purpose of this study is to design a novel effective algorithm for further improving the performance of glioma subtype classification using multimodal MRI images.MethodMRI images of four modalities for 221 glioma patients were collected from Computational Precision Medicine: Radiology-Pathology 2020 challenge, including T1, T2, T1ce, and fluid-attenuated inversion recovery (FLAIR) MRI images, to classify astrocytoma, oligodendroglioma, and glioblastoma. We proposed a multimodal MRI image decision fusion-based network for improving the glioma classification accuracy. First, the MRI images of each modality were input into a pre-trained tumor segmentation model to delineate the regions of tumor lesions. Then, the whole tumor regions were centrally clipped from original MRI images followed by max–min normalization. Subsequently, a deep learning-based network was designed based on a unified DenseNet structure, which extracts features through a series of dense blocks. After that, two fully connected layers were used to map the features into three glioma subtypes. During the training stage, we used the images of each modality after tumor segmentation to train the network to obtain its best accuracy on our testing set. During the inferring stage, a linear weighted module based on a decision fusion strategy was applied to assemble the predicted probabilities of the pre-trained models obtained in the training stage. Finally, the performance of our method was evaluated in terms of accuracy, area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), etc.ResultsThe proposed method achieved an accuracy of 0.878, an AUC of 0.902, a sensitivity of 0.772, a specificity of 0.930, a PPV of 0.862, an NPV of 0.949, and a Cohen’s Kappa of 0.773, which showed a significantly higher performance than existing state-of-the-art methods.ConclusionCompared with current studies, this study demonstrated the effectiveness and superiority in the overall performance of our proposed multimodal MRI image decision fusion-based network method for glioma subtype classification, which would be of enormous potential value in clinical practice.

show abstract

Generative Adversarial Networks to Synthesize Missing T1 and FLAIR MRI Sequences for Use in a Multisequence Brain Tumor Segmentation Model

Cited by 74 publications

References 26 publications

Deep learning‐based convolutional neural network for intramodality brain MRI synthesis

Deep learning‐based convolutional neural network for intramodality brain MRI synthesis

Cycle-consistent adversarial networks improves generalizability of radiomics model in grading meningiomas on external validation

Multimodal MRI Image Decision Fusion-Based Network for Glioma Classification

Contact Info

Product

Resources

About