MGMDcGAN: Medical Image Fusion Using Multi-Generator Multi-Discriminator Conditional Generative Adversarial Network

Huang, Jun; Le, Zhuliang; Ma, Yong; Fan, Fan; Zhang, Hao; Yang, Lei

doi:10.1109/access.2020.2982016

Cited by 62 publications

(19 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some other studies explored different architectures involving multiple generators 54, 55 or multiple discriminators 56,57 to overcome the mode collapsing problem. Another recent work adopted multiple generators and multiple discriminators for medical image fusion 58 . In this study, the purpose of distributed multiple discriminators is to learn from multi-institutional and heterogeneous data sets.…”

Section: Discussionmentioning

confidence: 99%

Federated Synthetic Learning from Multi-institutional and Heterogeneous Medical Data

Chang

Yan²,

et al. 2021

Preprint

View full text Add to dashboard Cite

Statistically and information-wise adequate data plays a critical role in training a robust deep learning model. However, collecting sufficient medical data to train a centralized model is still challenging due to various constraints such as privacy regulations and security. In this work, we develop a novel privacy-preserving federated-discriminator GAN, named FedD-GAN, that can learn and synthesize high-quality and various medical images regardless of their type, from heterogeneous datasets residing in multiple data centers whose data cannot be transferred or shared. We trained and evaluated FedD-GAN on three essential classes of medical data, each involving different types of medical images: cardiac CTA, brain MRI, and histopathology. We show that the synthesized images using our method have better quality than using a standard federated learning method and are realistic and accurate enough to train accurate segmentation models in downstream tasks. The segmentation model trained on the synthetic data only is comparable to that trained on an all-in-one real-image dataset shared from multiple data centers if possible. FedD-GAN can learn to generate a scalable and diverse synthetic database without compromising data privacy. This synthetic database could help to boost machine learning techniques in medical data analytics.

show abstract

Section: Discussionmentioning

confidence: 99%

Federated Synthetic Learning from Multi-institutional and Heterogeneous Medical Data

Chang

Yan²,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Ma et al introduced a novel method based on an end-to-end model of a generative adversarial network (GAN) [47]; this method was successfully applied to the fusion of infrared and visible images. Similarly, Huang et al introduced a Wasserstein generative adversarial networks to color medical image fusion [48], in which a generator and two discriminators are employed to form a fused image. This method enhances the structure information and prevents the functional information from being weakened.…”

Section: Deep Learning-based Methodsmentioning

confidence: 99%

Two-Scale Multimodal Medical Image Fusion Based on Guided Filtering and Sparse Representation

2020

View full text Add to dashboard Cite

Medical image fusion techniques primarily integrate the complementary features of different medical images to acquire a single composite image with superior quality, reducing the uncertainty of lesion analysis. However, the simultaneous extraction of more salient features and less meaningless details from medical images by using multi-scale transform methods is a challenging task. This study presents a two-scale fusion framework for multimodal medical images to overcome the aforementioned limitation. In this framework, a guided filter is used to decompose source images into the base and detail layers to roughly separate the two characteristics of source images, namely, structural information and texture details. To effectively preserve most of the structural information, the base layers are fused using the combined Laplacian pyramid and sparse representation rule, in which an image patch selection-based dictionary construction scheme is introduced to exclude the meaningless patches from the source images and enhance the sparse representation capability of the pyramid-decomposed low-frequency layer. The detail layers are subsequently merged using a guided filtering-based approach, which enhances contrast level via noise filtering as much as possible. The fused base and detail layers are reconstructed to generate the fused image. We experimentally verify the superiority of the proposed method by using two basic fusion schemes and conducting comparison experiments on nine pairs of medical images from diverse modalities. The comparison of the fused results in terms of visual effect and objective assessment demonstrates that the proposed method provides better visual effect with an improved objective measurement because it effectively preserves meaningful salient features without producing abnormal details. INDEX TERMS Medical image fusion, guided filtering, sparse representation, salient features, meaningless details.

show abstract

“…Since Gan was first proposed in 2014 to the present, regardless of the image size or image fidelity, the performance of GAN has been constantly improving and breakthrough. The main application of GAN is in image processing, including image-to-image translation, using GAN for image restoration [32,33], improving image resolution [34], medical image fusion [35,36] etc. Recently, the application of Gan in medical imaging has gradually increased and achieved remarkable results [37].…”

Section: Generative Adversarial Network (Gan)mentioning

confidence: 99%

Multi-Disease Prediction Based on Deep Learning: A Survey

Xie¹,

Yu²,

Lv³

2021

Computer Modeling in Engineering &Amp; Sciences

161

104

View full text Add to dashboard Cite

In recent years, the development of artificial intelligence (AI) and the gradual beginning of AI's research in the medical field have allowed people to see the excellent prospects of the integration of AI and healthcare. Among them, the hot deep learning field has shown greater potential in applications such as disease prediction and drug response prediction. From the initial logistic regression model to the machine learning model, and then to the deep learning model today, the accuracy of medical disease prediction has been continuously improved, and the performance in all aspects has also been significantly improved. This article introduces some basic deep learning frameworks and some common diseases, and summarizes the deep learning prediction methods corresponding to different diseases. Point out a series of problems in the current disease prediction, and make a prospect for the future development. It aims to clarify the effectiveness of deep learning in disease prediction, and demonstrates the high correlation between deep learning and the medical field in future development. The unique feature extraction methods of deep learning methods can still play an important role in future medical research.

show abstract

MGMDcGAN: Medical Image Fusion Using Multi-Generator Multi-Discriminator Conditional Generative Adversarial Network

Cited by 62 publications

References 46 publications

Federated Synthetic Learning from Multi-institutional and Heterogeneous Medical Data

Federated Synthetic Learning from Multi-institutional and Heterogeneous Medical Data

Two-Scale Multimodal Medical Image Fusion Based on Guided Filtering and Sparse Representation

Multi-Disease Prediction Based on Deep Learning: A Survey

Contact Info

Product

Resources

About