Deep learning based FACS Action Unit occurrence and intensity estimation

Gudi, Amogh; Taşlı, Hamdi; Uyl, Tim M. den; Maroulis, Andreas

doi:10.1109/fg.2015.7284873

Cited by 144 publications

(108 citation statements)

References 10 publications

Supporting

Mentioning

104

Contrasting

Unclassified

Order By: Relevance

“…4 shows the weighted average performance on SEMAINE and BP4D dataset. In this Fig. we compare the performance of our method with all other approaches ( [28], [2], [9], [23]) of the partcipants of FERA-2015 Challenge. In this Fig. we can see that our method significantly outperforms other approaches on the FERA-2015 Challenge dataset.…”

Section: Discussionmentioning

confidence: 99%

“…This CNN consisted of 3 different data streams corresponding to the three scales which are connected to each other only at the fully connected layer of the network. Gudi et al [9] used a deep CNN consisting of 3 convolutional layers, 1 sub-sampling layer and 1 fully connected layer to predict the occurrence and intensity of Facial AUs. A similar architecture was used by Tang [21], but replacing the softmax objective function with L2-SVM.…”

Section: Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning the dynamic appearance and shape of facial action units

Jaiswal

Valstar

2016

2016 IEEE Winter Conference on Applications of Computer Vision (WACV)

138

View full text Add to dashboard Cite

Spontaneous facial expression recognition under uncontrolled conditions is a hard task. It depends on multiple factors including shape, appearance and dynamics of the facial features, all of which are adversely affected by environmental noise and low intensity signals typical of such conditions. In this work, we present a novel approach to Facial Action Unit detection using a combination of Convolutional and Bi-directional Long Short-Term Memory Neural Networks (CNN-BLSTM), which jointly learns shape, appearance and dynamics in a deep learning manner. In addition, we introduce a novel way to encode shape features using binary image masks computed from the locations of facial landmarks. We show that the combination of dynamic CNN features and Bi-directional Long Short-Term Memory excels at modelling the temporal information. We thoroughly evaluate the contributions of each component in our system and show that it achieves state-of-the-art performance on the FERA-2015 Challenge dataset.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Previous Workmentioning

confidence: 99%

Deep learning the dynamic appearance and shape of facial action units

Jaiswal

Valstar

2016

2016 IEEE Winter Conference on Applications of Computer Vision (WACV)

138

View full text Add to dashboard Cite

show abstract

“…For example, [69] attained reasonable performance on the FERA 2015 challenge using standard deeply learnt features, and Jaiswal et althat presented a novel deep learning-based representation encoding dynamic appearance and face shape [79] attained state-of-the-art results on that database.…”

Section: Deeply Learnt Featuresmentioning

confidence: 99%

“…While the scalability of ANN to large datasets is one of its strongest aspects, the amount of available data for AU analysis remains relatively scarce. It would nonetheless be interesting to study their performance given the recent resurgence of ANN, specially as some promising works have recently appeared [69], [79]. Boosting algorithms, such as AdaBoost and GentleBoost, have been a common choice for AU recognition, e.g.…”

Section: Analysis Of Individual Aumentioning

confidence: 99%

Automatic Analysis of Facial Actions: A Survey

Martínez

Valstar

Jiang

et al. 2019

IEEE Trans. Affective Comput.

238

153

View full text Add to dashboard Cite

Abstract-As one of the most comprehensive and objective ways to describe facial expressions, the Facial Action Coding System (FACS) has recently received significant attention. Over the past 30 years, extensive research has been conducted by psychologists and neuroscientists on various aspects of facial expression analysis using FACS. Automating FACS coding would make this research faster and more widely applicable, opening up new avenues to understanding how we communicate through facial expressions. Such an automated process can also potentially increase the reliability, precision and temporal resolution of coding. This paper provides a comprehensive survey of research into machine analysis of facial actions. We systematically review all components of such systems: pre-processing, feature extraction and machine coding of facial actions. In addition, the existing FACS-coded facial expression databases are summarised. Finally, challenges that have to be addressed to make automatic facial action analysis applicable in real-life situations are extensively discussed. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the future of machine recognition of facial actions: what are the challenges and opportunities that researchers in the field face.

show abstract

“…To tackle these challenges, recent deep learning approaches to AU analysis have adopted some heuristical techniques such as iterative balanced batch, simple data augmentation, training on multiple databases, etc. [5] [2] [3], which will alleviate but not fully solve difficulties with limited AU label data. This provides the motivation for automatic facial expression editing / synthesis, such that desired facial expressions corresponding to given AU labels can be generated, in order to create a large-scale facial expression dataset with accurate and comprehensively diverse AU labels.…”

Section: Introductionmentioning

confidence: 99%

Conditional adversarial synthesis of 3D facial action units

et al. 2019

View full text Add to dashboard Cite

Employing deep learning-based approaches for finegrained facial expression analysis, such as those involving the estimation of Action Unit (AU) intensities, is difficult due to the lack of a large-scale dataset of real faces with sufficiently diverse AU labels for training. In this paper, we consider how AU-level facial image synthesis can be used to substantially augment such a dataset. We propose an AU synthesis framework that combines the well-known 3D Morphable Model (3DMM), which intrinsically disentangles expression parameters from other face attributes, with models that adversarially generate 3DMM expression parameters conditioned on given target AU labels, in contrast to the more conventional approach of generating facial images directly. In this way, we are able to synthesize new combinations of expression parameters and facial images from desired AU labels. Extensive quantitative and qualitative results on the benchmark DISFA dataset demonstrate the effectiveness of our method on 3DMM facial expression parameter synthesis and data augmentation for deep learning-based AU intensity estimation.

show abstract

Deep learning based FACS Action Unit occurrence and intensity estimation

Cited by 144 publications

References 10 publications

Deep learning the dynamic appearance and shape of facial action units

Deep learning the dynamic appearance and shape of facial action units

Automatic Analysis of Facial Actions: A Survey

Conditional adversarial synthesis of 3D facial action units

Contact Info

Product

Resources

About