Can Segmentation Models Be Trained with Fully Synthetically Generated Data?

Fernandez, Virginia; Pinaya, Walter Hugo Lopez; Borges, Pedro; Tudosiu, Petru-Daniel; Graham, Mark S.; Vercauteren, Tom; Cardoso, M. Jorge

doi:10.1007/978-3-031-16980-9_8

Cited by 22 publications

(12 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is in line with previously reported findings that synthesized data from GANs can improve the performance of medical AI models. Although the addition of real images improves performance more than the addition of the synthetic images, the synthetic images have the added benefit of minimizing privacy concerns …”

Section: Discussionmentioning

confidence: 99%

Generation of a Melanoma and Nevus Data Set From Unstandardized Clinical Photographs on the Internet

Cho,

Navarrete-Dechent,

Daneshjou

et al. 2023

JAMA Dermatol

View full text Add to dashboard Cite

ImportanceArtificial intelligence (AI) training for diagnosing dermatologic images requires large amounts of clean data. Dermatologic images have different compositions, and many are inaccessible due to privacy concerns, which hinder the development of AI.ObjectiveTo build a training data set for discriminative and generative AI from unstandardized internet images of melanoma and nevus.Design, Setting, and ParticipantsIn this diagnostic study, a total of 5619 (CAN5600 data set) and 2006 (CAN2000 data set; a manually revised subset of CAN5600) cropped lesion images of either melanoma or nevus were semiautomatically annotated from approximately 500 000 photographs on the internet using convolutional neural networks (CNNs), region-based CNNs, and large mask inpainting. For unsupervised pretraining, 132 673 possible lesions (LESION130k data set) were also created with diversity by collecting images from 18 482 websites in approximately 80 countries. A total of 5000 synthetic images (GAN5000 data set) were generated using the generative adversarial network (StyleGAN2-ADA; training, CAN2000 data set; pretraining, LESION130k data set).Main Outcomes and MeasuresThe area under the receiver operating characteristic curve (AUROC) for determining malignant neoplasms was analyzed. In each test, 1 of the 7 preexisting public data sets (total of 2312 images; including Edinburgh, an SNU subset, Asan test, Waterloo, 7-point criteria evaluation, PAD-UFES-20, and MED-NODE) was used as the test data set. Subsequently, a comparative study was conducted between the performance of the EfficientNet Lite0 CNN on the proposed data set and that trained on the remaining 6 preexisting data sets.ResultsThe EfficientNet Lite0 CNN trained on the annotated or synthetic images achieved higher or equivalent mean (SD) AUROCs to the EfficientNet Lite0 trained using the pathologically confirmed public data sets, including CAN5600 (0.874 [0.042]; P = .02), CAN2000 (0.848 [0.027]; P = .08), and GAN5000 (0.838 [0.040]; P = .31 [Wilcoxon signed rank test]) and the preexisting data sets combined (0.809 [0.063]) by the benefits of increased size of the training data set.Conclusions and RelevanceThe synthetic data set in this diagnostic study was created using various AI technologies from internet images. A neural network trained on the created data set (CAN5600) performed better than the same network trained on preexisting data sets combined. Both the annotated (CAN5600 and LESION130k) and synthetic (GAN5000) data sets could be shared for AI training and consensus between physicians.

show abstract

Section: Discussionmentioning

confidence: 99%

Generation of a Melanoma and Nevus Data Set From Unstandardized Clinical Photographs on the Internet

Cho,

Navarrete-Dechent,

Daneshjou

et al. 2023

JAMA Dermatol

View full text Add to dashboard Cite

show abstract

“…Fernandez et al [101] introduce a generative model, named brainSPADE, for synthesizing labeled brain MRI images that can be used for training segmentation models. The model combines a diffusion model with a VAE-GAN, with the GAN component particularly utilizing SPADE normalization to incorporate the segmentation mask.…”

Section: Diffusion Modelsmentioning

confidence: 99%

Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review

2023

View full text Add to dashboard Cite

Deep learning has become a popular tool for medical image analysis, but the limited availability of training data remains a major challenge, particularly in the medical field where data acquisition can be costly and subject to privacy regulations. Data augmentation techniques offer a solution by artificially increasing the number of training samples, but these techniques often produce limited and unconvincing results. To address this issue, a growing number of studies have proposed the use of deep generative models to generate more realistic and diverse data that conform to the true distribution of the data. In this review, we focus on three types of deep generative models for medical image augmentation: variational autoencoders, generative adversarial networks, and diffusion models. We provide an overview of the current state of the art in each of these models and discuss their potential for use in different downstream tasks in medical imaging, including classification, segmentation, and cross-modal translation. We also evaluate the strengths and limitations of each model and suggest directions for future research in this field. Our goal is to provide a comprehensive review about the use of deep generative models for medical image augmentation and to highlight the potential of these models for improving the performance of deep learning algorithms in medical image analysis.

show abstract

“…brainSPADE [43] proposes a generative model for synthesizing labeled brain MRI images that can be used for training segmentation models. brainSPADE is composed of a label generator and an image generator sub-model.…”

Section: Segmentationmentioning

confidence: 99%

“…A spatial VAE decoder then constructs the artificial segmentation map via the latent space. In the image generator, Fernandez et al [43] take advantage of SPADE [110], a VAE-GAN model, to build a style latent space from the input arbitrary style and use it with the artificial segmentation map to decode the output image. nnU-Net [111] was leveraged to examine the performance.…”

Section: Segmentationmentioning

confidence: 99%

“…• Generalization to other types of noise distributions [53] • Facilitating diffusion process via implicit guidance [54] • Acceleration improvement using trainingfree DDIM sampling [55,28] • Guiding diffusion process via classifier guidance [28] • Conditional DDPMs [46,32,36,50,135,57] • Generating synthetic segmentation datasets [43,44] • Cross-modality translation [34] • Multi-modal conversion [32,34] • Exploiting K-space parameter-free guidance [38] • Accelerate MC sampling using coarse-tofine sampling [38] • Adversarial learning in the reverse diffusion process [39,34] • 3D reconstruction from 2D images [50] • Conditioning on medical meta-data [48] • Using LDM to enhance the training and sampling efficiency [48,59] • DDPMs for histopathology images [49] • 3D medical image generation [51] • Using deformation fields for medical image generation [134,52] • DDPMs in skin image adversarial attacks [56] Noise Conditioned Score Networks (NCSNs) 1 CSGM-MRI-Langevin [96] 2 Self-Score [41] In this algorithm [61], creating samples requires solving the Langevin dynamics equation. However, this equation mandates the solution of the gradient of the log density w.r.t.…”

Section: Comparative Overviewmentioning

confidence: 99%

See 1 more Smart Citation

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

Kazerouni¹,

Aghdam²,

Heidari³

et al. 2022

Preprint

View full text Add to dashboard Cite

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical image analysis. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including segmentation, anomaly detection, image-to-image translation, 2/3D generation, reconstruction, denoising, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub 1 . We aim to update the relevant latest papers within it regularly.

show abstract

Can Segmentation Models Be Trained with Fully Synthetically Generated Data?

Cited by 22 publications

References 22 publications

Generation of a Melanoma and Nevus Data Set From Unstandardized Clinical Photographs on the Internet

Generation of a Melanoma and Nevus Data Set From Unstandardized Clinical Photographs on the Internet

Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

Contact Info

Product

Resources

About