Towards Discovery and Attribution of Open-world GAN Generated Images

Girish, Sharath; Suri, Saksham; Saketh, Rambhatla,; Shrivastava, Abhinav

doi:10.1109/iccv48922.2021.01383

Cited by 28 publications

(10 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model attribution. Model attribution aims to identify the specific model behind a synthetic image or video [18]. With an increasing number of GAN models introduced in image manipulation, several studies [16,22,17,18,23] have investigated GAN model attribution in fake images.…”

Section: Related Workmentioning

confidence: 99%

“…Model attribution aims to identify the specific model behind a synthetic image or video [18]. With an increasing number of GAN models introduced in image manipulation, several studies [16,22,17,18,23] have investigated GAN model attribution in fake images. These methods propose to extract unique artificial fingerprints left by different GAN models and perform multi-class classification for GAN model attribution.…”

Section: Related Workmentioning

confidence: 99%

“…The model attribution problem has been considered in recent studies [16,17,18] in the case of Generative Adversarial Network (GAN)-based models. However, the model attribution method designed for GAN images cannot be extended to face-swap Deepfakes videos.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Model Attribution of Face-swap Deepfake Videos

Jia¹,

Li²,

Lyu³

2022

Preprint

View full text Add to dashboard Cite

AI-created face-swap videos, commonly known as Deepfakes, have attracted wide attention as powerful impersonation attacks. Existing research on Deepfakes mostly focuses on binary detection to distinguish between real and fake videos. However, it is also important to determine the specific generation model for a fake video, which can help attribute it to the source for forensic investigation. In this paper, we fill this gap by studying the model attribution problem of Deepfake videos. We first introduce a new dataset with DeepFakes from Different Models (DFDM) based on several Autoencoder models. Specifically, five generation models with variations in encoder, decoder, intermediate layer, input resolution, and compression ratio have been used to generate a total of 6, 450 Deepfake videos based on the same input. Then we take Deepfakes model attribution as a multiclass classification task and propose a spatial and temporal attention based method to explore the differences among Deepfakes in the new dataset. Experimental evaluation shows that most existing Deepfakes detection methods failed in Deepfakes model attribution, while the proposed method achieved over 70% accuracy on the high-quality DFDM dataset 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Model Attribution of Face-swap Deepfake Videos

Jia¹,

Li²,

Lyu³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Besides of the featurelearning methods mentioned-above, recent works also utilize various training strategies to improve the generalization, such as transfer learning, incremental learning, contrastive learning, representation learning, and self-supervised learning [14]- [17], [39]- [41], etc. Moreover, [42] proposes a model to learn fingerprints commonly shared by GANs for open-world GAN image detection based on out-of-distribution detection. Data augmentation has been used to improve generalization ability and robustness in [1], [43].…”

Section: B Face Forgery Detectionmentioning

confidence: 99%

GLFF: Global and Local Feature Fusion for Face Forgery Detection

Jiang¹,

Jia²,

Cai³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the rapid development of deep generative models (such as Generative Adversarial Networks and Autoencoders), AI-synthesized images of the human face are now of such high quality that humans can hardly distinguish them from pristine ones. Although existing detection methods have shown high performance in specific evaluation settings, e.g., on images from seen models or on images without real-world postprocessing, they tend to suffer serious performance degradation in real-world scenarios where testing images can be generated by more powerful generation models or combined with various post-processing operations. To address this issue, we propose a Global and Local Feature Fusion (GLFF) framework to learn rich and discriminative representations by combining multiscale global features from the whole image with refined local features from informative patches for face forgery detection. GLFF fuses information from two branches: the global branch to extract multi-scale semantic features and the local branch to select informative patches for detailed local artifacts extraction. Due to the lack of a face forgery dataset simulating real-world applications for evaluation, we further create a challenging face forgery dataset, named DeepFakeFaceForensics (DF 3 ), which contains 6 state-of-the-art generation models and a variety of post-processing techniques to approach the real-world scenarios. Experimental results demonstrate the superiority of our method to the state-of-the-art methods on the proposed DF 3 dataset and three other open-source datasets.

show abstract

“…Moreover, GAN-generated data can embed backdoors that can trigger unwanted model outputs [12]. Several forensic techniques can trace the source of generated data, addressing the concerns about its quality [13,14,15,16,17,18]. However, these techniques only work at the data level.…”

Section: Introductionmentioning

confidence: 99%

Detection and Attribution of Models Trained on Generated Data

Han,

Salem,

et al. 2024

ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have become widely used in model training, as they can improve performance and/or protect sensitive information by generating data. However, this also raises potential risks, as malicious GANs may compromise or sabotage models by poisoning their training data. Therefore, it is important to verify the origin of a model's training data for accountability purposes. In this work, we take the first step in the forensic analysis of models trained on GAN-generated data. Specifically, we first detect whether a model is trained on GAN-generated or real data. We then attribute these models, trained on GANgenerated data, to their respective source GANs. We conduct extensive experiments on three datasets, using four popular GAN architectures and four common model architectures. Empirical results show the remarkable performance of our detection and attribution methods. Furthermore, we conduct a more in-depth study and reveal that models trained on various data sources exhibit different decision boundaries and behaviours.

show abstract

Towards Discovery and Attribution of Open-world GAN Generated Images

Cited by 28 publications

References 37 publications

Model Attribution of Face-swap Deepfake Videos

Model Attribution of Face-swap Deepfake Videos

GLFF: Global and Local Feature Fusion for Face Forgery Detection

Detection and Attribution of Models Trained on Generated Data

Contact Info

Product

Resources

About