When do GANs replicate? On the choice of dataset size

Feng, Qianli; Guo, Chenqi; Benitez-Quiroz, Fabian; Martı́nez, Aleix M.

doi:10.1109/iccv48922.2021.00663

Cited by 30 publications

(22 citation statements)

References 24 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, numerous metrics exist to measure similarity with the training data [32,3], the extent of mode collapse [61,15], and the impact of individual training samples [4,75]. Moreover, other work provides insights into when and why GANs may replicate training examples [50,26], as well as how to mitigate such effects [50]. Our work extends these lines of inquiry to conditional diffusion models, where we measure novelty by computing how frequently models regenerate training instances when provided with textual prompts.…”

Section: Related Workmentioning

confidence: 99%

Extracting Training Data from Diffusion Models

Carlini¹,

Hayes²,

Nasr³

et al. 2023

Preprint

View full text Add to dashboard Cite

Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images. In this work, we show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from stateof-the-art models, ranging from photographs of individual people to trademarked company logos. We also train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy. Overall, our results show that diffusion models are much less private than prior generative models such as GANs, and that mitigating these vulnerabilities may require new advances in privacy-preserving training.

show abstract

Section: Related Workmentioning

confidence: 99%

Extracting Training Data from Diffusion Models

Carlini¹,

Hayes²,

Nasr³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Learningbased models can also be vulnerable to inference attacks on models aiming to leak sensitive information about training data [22]. Differential privacy between training and synthetic samples in adversarial models substantially improves for large and diverse training datasets as encountered in FL settings [67]. Furthermore, FedGIMP uses a shared generator without direct access to data and unshared discriminators that are not communicated [68].…”

Section: Discussionmentioning

confidence: 99%

Federated Learning of Generative Image Priors for MRI Reconstruction

Elmas

Dar

Korkmaz

et al. 2023

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Multi-institutional efforts can facilitate training of deep MRI reconstruction models, albeit privacy risks arise during cross-site sharing of imaging data. Federated learning (FL) has recently been introduced to address privacy concerns by enabling distributed training without transfer of imaging data. Existing FL methods employ conditional reconstruction models to map from undersampled to fully-sampled acquisitions via explicit knowledge of the accelerated imaging operator. Since conditional models generalize poorly across different acceleration rates or sampling densities, imaging operators must be fixed between training and testing, and they are typically matched across sites. To improve patient privacy, performance and flexibility in multi-site collaborations, here we introduce Federated learning of Generative IMage Priors (FedGIMP) for MRI reconstruction. FedGIMP leverages a two-stage approach: cross-site learning of a generative MRI prior, and prior adaptation following injection of the imaging operator. The global MRI prior is learned via an unconditional adversarial model that synthesizes high-quality MR images based on latent variables. A novel mapper subnetwork produces site-specific latents to maintain specificity in the prior. During inference, the prior is first combined with subject-specific imaging operators to enable reconstruction, and it is then adapted to individual cross-sections by minimizing a data-consistency loss. Comprehensive experiments on multi-institutional datasets clearly demonstrate enhanced performance of FedGIMP against both centralized and FL methods based on conditional models.

show abstract

“…Learning-based models can also be vulnerable to inference attacks on models aiming to leak sensitive information about training data [21]. Differential privacy between training and synthetic samples in adversarial models substantially improves for large and diverse training datasets as encountered in FL settings [64]. Furthermore, FedGIMP uses a shared generator without direct access to data and unshared discriminators that are not communicated [65].…”

Section: Discussionmentioning

confidence: 99%

Federated Learning of Generative Image Priors for MRI Reconstruction

Elmas¹,

Dar²,

Korkmaz³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multi-institutional efforts can facilitate training of deep MRI reconstruction models, albeit privacy risks arise during cross-site sharing of imaging data. Federated learning (FL) has recently been introduced to address privacy concerns by enabling distributed training without transfer of imaging data. Existing FL methods for MRI reconstruction employ conditional models to map from undersampled to fully-sampled acquisitions via explicit knowledge of the imaging operator. Since conditional models generalize poorly across different acceleration rates or sampling densities, imaging operators must be fixed between training and testing, and they are typically matched across sites. To improve generalization and flexibility in multi-institutional collaborations, here we introduce a novel method for MRI reconstruction based on Federated learning of Generative IMage Priors (FedGIMP). FedGIMP leverages a two-stage approach: cross-site learning of a generative MRI prior, and subject-specific injection of the imaging operator. The global MRI prior is learned via an unconditional adversarial model that synthesizes high-quality MR images based on latent variables. Specificity in the prior is preserved via a mapper subnetwork that produces sitespecific latents. During inference, the prior is combined with subject-specific imaging operators to enable reconstruction, and further adapted to individual test samples by minimizing data-consistency loss. Comprehensive experiments on multi-institutional datasets clearly demonstrate enhanced generalization performance of FedGIMP against site-specific and federated methods based on conditional models, as well as traditional reconstruction methods.

show abstract

When do GANs replicate? On the choice of dataset size

Cited by 30 publications

References 24 publications

Extracting Training Data from Diffusion Models

Extracting Training Data from Diffusion Models

Federated Learning of Generative Image Priors for MRI Reconstruction

Federated Learning of Generative Image Priors for MRI Reconstruction

Contact Info

Product

Resources

About