Seeing What a GAN Cannot Generate

This paper studies reconstruction of human body shape and pose from a single-view image. While most of current work attempts to regress parameters of human body model such as Skinned Multi-Person Linear Model (SMPL) and Hand Model with Articulated and Non-rigid Deformations (MANO), these parametric approaches underperform compared to non-parametric approaches. Due to the lack of the spatial relationship in the input image, the parametric approaches are hardly used to reconstruct the human body precisely. Besides, the rotation parameter regression is a complex task in parametric approaches. Therefore, we introduce a novel graph convolutional neural network (Graph CNN)-based framework for estimating a non-parametric mesh model. Our key innovation is that the proposed model is trained in a generative adversarial manner. Firstly, Graph CNN utilizes mesh topology to capture integral information of the full 3D human shape and then generate a more smooth and high-quality human mesh model. Secondly, the discriminator in our network acts as a supervisor to specify whether a human shape and pose are real or not. The generator is encouraged to generate human body mesh that is close to the manifold of the real human mesh distribution. Extensive experimental results demonstrate the effectiveness of our proposed framework. In contrast to the state-of-the-art methods, our method can achieve better performance in human shape and pose estimation.

Section: Methodsmentioning

confidence: 99%

Graph Convolutional Adversarial Network for Human Body Pose and Mesh Estimation

Huang

2020

“…Most of the recent works include GANs generated images and videos and detecting counterfeits using GANs itself from both GANs generated deepfakes and GANs detection of deepfakes. Bau et al [38] concluded that GANs have limited role and capacity in the generation and analyzed that the pre-trained GANs model is unable to grab the image structures from the given datasets. Having limitations in a generation, it is obvious that the GANs model is not so reliable in detecting deepfakes alone.…”

Section: Artifacts From Gansmentioning

confidence: 99%

An Exploratory Analysis on Visual Counterfeits Using Conv-LSTM Hybrid Architecture

Hashmi

Ashish²,

Keskar

et al. 2020

In recent years, with the advancements in the Deep Learning realm, it has been easy to create and generate synthetically the face swaps from GANs and other tools, which are very realistic, leaving few traces which are unclassifiable by human eyes. These are known as 'DeepFakes' and most of them are anchored in video formats. Such realistic fake videos and images are used to create a ruckus and affect the quality of public discourse on sensitive issues; defaming one's profile, political distress, blackmailing and many more fake cyber terrorisms are envisioned. This work proposes a microscopic-typo comparison of video frames. This temporal-detection pipeline compares very minute visual traces on the faces of real and fake frames using Convolutional Neural Network (CNN) and stores the abnormal features for training. A total of 512 facial landmarks were extracted and compared. Parameters such as eye-blinking lip-synch; eyebrows movement, and position, are few main deciding factors that classify into real or counterfeit visual data. The Recurrent Neural Network (RNN) pipeline learns based on these features-fed inputs and then evaluates the visual data. The model was trained with the network of videos consisting of their real and fake, collected from multiple websites. The proposed algorithm and designed network set a new benchmark for detecting the visual counterfeits and show how this system can achieve competitive results on any fake generated video or image. INDEX TERMS DeepFakes, Generative Adversarial Network (GANs), Facial landmarks, Convolutional Neural Networks (CNN), Recurrent Neural Network (RNN), Visual Counterfeits.

“…Although WGAN-GP [34] overcame the mode collapse and training convergence issues, the use of gradient penalty weakened the representation capacity of GAN [40].…”

Section: Introductionmentioning

confidence: 99%

“…To deal with the problem and enlightened by the results [38], [40], here we propose a residual-CNN-block generator and discriminator for noise learning with the least squares [41]. The structural similarity (SSIM) [36] and L1 losses integrated into the overall objective function.…”

Section: Introductionmentioning

confidence: 99%

Low-Dose CT Image Denoising Using a Generative Adversarial Network With a Hybrid Loss Function for Noise Learning

Wei

Feng

et al. 2020

Potential risk of X-ray radiation from computed tomography (CT) has been a concern of the public. However, simply decreasing the dose will degrade quality of the CT images and compromise diagnostic performance. In this paper, we propose a noise learning generative adversarial network coupling with least squares, structural similarity and L1 losses for low-dose CT denoising. In our method, noise distributed in the input low-dose CT image is learned by the generator network and then subtracted from the input to generate the final denoised version. The denoised CT images are penalized by the least squares loss function, and they are pulled toward boundary of the decision even though they are classified as normal-dose CT. Least squares stabilize the training process without regularization. Structural similarity and L1 losses are utilized to keep textural details and sharpness of the denoised CT images respectively. Experiments and results show that our method can effectively suppress noise and remove artifacts compared with the state-ofthe-art methods. The texture statistical properties, which include mean, standard deviation, uniformity, and entropy, further confirm that the generated noise-reduced CT image is as close as to that of the normal-dose counterpart. INDEX TERMS Deep learning, generative adversarial network, least squares, low-dose CT, denoising.