A Closer Look at Fourier Spectrum Discrepancies for CNN-generated Images Detection

Keshigeyan, Chandrasegaran,; Tran, Ngoc-Trung; Cheung, Ngai-Man

doi:10.1109/cvpr46437.2021.00712

Cited by 52 publications

(29 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From now on, we refer to SR by azimuthal integration [21] as the AI loss, and to [24] as the Watson-DFT loss. In [17], Chandrasegaran et al argue that the spectral discrepancies are not inherent to the neural network, but an artifact from the up-sampling procedure. They show promising results by replacing the last transposed convolution layer with either zero-insert scaling, nearest interpolation, or bilinear interpolation followed by traditional convolution.…”

Section: Related Workmentioning

confidence: 99%

“…The alternative approach is to split the transformation in two: up-sampling by interpolation and convolution. The transposed convolution operation is known to have several shortcomings, such as high-frequency discrepancies and checkerboard artifacts [17,32]. Chandrasegaran et al [17] propose multiple ways to perform the up-sampling in the last layer of a generator network.…”

Section: Up-sampling and Transposed Convolutionmentioning

confidence: 99%

“…In this work we show that a simple frequency-aware loss that forces the generative model to focus on agreement of the overall spectral content of the data is equally effective, and sometimes better than the current state-of-the-art-in SR [21,24]. Furthermore, we consider the effects of replacing the upsampling operation in the last layer, similar to [17,26]. This allows us to evaluate the impact of the up-sampling operation both with and without spectral regularization.…”

Section: Introductionmentioning

confidence: 96%

“…A growing body of research has investigated these findings, see e.g. [16,17,18,19], and ways to utilize this in e.g. deep-fake detection [20,21,22,23].…”

Section: Introductionmentioning

confidence: 99%

“…deep-fake detection [20,21,22,23]. Others propose different ways to resolve, work around, or reduce the effects of a bias towards the low-frequency components [21,24,22,25,24,17,26]. Another partial explanation for the discrepancy in the frequency content of generated images is the transposed convolution operation used in the up-sampling components of generative models.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Simpler is better: spectral regularization and up-sampling techniques for variational autoencoders

Björk¹,

Myhre²,

Johansen³

2022

Preprint

View full text Add to dashboard Cite

Full characterization of the spectral behavior of generative models based on neural networks remains an open issue. Recent research has focused heavily on generative adversarial networks and the high-frequency discrepancies between real and generated images. The current solution to avoid this is to either replace transposed convolutions with bilinear upsampling or add a spectral regularization term in the generator. It is well known that Variational Autoencoders (VAEs) also suffer from these issues.In this work, we propose a simple 2D Fourier transformbased spectral regularization loss for the VAE and show that it can achieve results equal to, or better than, the current state-ofthe-art in frequency-aware losses for generative models. In addition, we experiment with altering the up-sampling procedure in the generator network and investigate how it influences the spectral performance of the model. We include experiments on synthetic and real data sets to demonstrate our results.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Up-sampling and Transposed Convolutionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

“…A growing body of research has investigated these findings, see e.g. [16,17,18,19], and ways to utilize this in e.g. deep-fake detection [20,21,22,23].…”

Section: Introductionmentioning

confidence: 99%