Local and Global Perception Generative Adversarial Network for Facial Expression Synthesis

Abstract

Facial expression synthesis has gained increasing attention with the development of Generative Adversarial Networks (GANs). However, it is still very challenging to generate high-quality facial expressions since the overlapping and blur commonly appear in the generated facial images especially in the regions with rich facial features such as eye and mouth. Generally, existing methods mainly consider the face as a whole in facial expression synthesis without paying specific attention to the characteristics of facial expressions. In fact, according to the physiological and psychological research, the differences of facial expressions often appear in crucial regions such as eye and mouth. Motivated by this observation, a novel end-toend facial expression synthesis method called Local and Global Perception Generative Adversarial Network (LGP-GAN) with a two-stage cascaded structure is proposed in this paper which is designed to extract and synthesize the details of the crucial facial regions. LGP-GAN can combine the generated results from the global network and local network into the corresponding facial expressions. In Stage I, LGP-GAN utilizes local networks to capture the local texture details of the crucial facial regions and generate local facial regions, which fully explores crucial facial region domain information in facial expressions. And then LGP-GAN uses a global network to learn the whole facial information in Stage II to generate the generate final facial expressions building upon local generated results from Stage I. We conduct qualitative and quantitative experiments on the commonly used public database to verify the effectiveness of the proposed method. Experimental results show the superiority of the proposed method over the state-of-the-art methods.

Is High-Fidelity Important for Human-like Virtual Avatars in Human Computer Interactions?

“…Other studies explore methods for applying high-fidelity motion to those virtual avatars to make them seemingly move actual humans. Motion recognition [57], motion tracking [58] and motion caption systems [59] play essential roles in the development of those techniques. Even though most tracking and recognition techniques do not directly transfer motion tracked from real humans to virtual avatars, still, those tracking techniques are qualified resources for applying reference to character motion design and can potentially fill the gap in current motion capture systems.…”

SARGAN: Spatial Attention-Based Residuals for Facial Expression Manipulation

“…GANs have proven successful for face-related tasks including facial attribute editing [3], [4], [5], [6] and face super-resolution [7], [8], [9], [10]. A substantial amount of work has also been done in the area of facial expression synthesis with the prevalence of GANs such as StarGAN [11], GANimation [12], Cascade-EF GAN [13], DAI2I [14] and LGP-GAN [15]. However, most of these methods including StarGAN, GANimation, Cascade-EF GAN and DAI2I need large training datasets.…”

A novel art gesture recognition model based on two channel region-based convolution neural network for explainable human-computer interaction understanding

“…When the main direction of the gesture obtained was inconsistent with the main direction of similar gestures in the training library, it was prone to error recognition. In view of the complexity of current gesture feature extraction methods and the low gesture recognition rate in complex background, the research and practice of references [28,29] found that convolutional neural network (CNN) had scale invariance to flip, pan and scale. It was better than other machine vision methods in gesture detection and recognition applications.…”