A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modeling

Jiang, Bihan; Valstar, Michel; Martínez, Brais; Pantić, Maja

doi:10.1109/tcyb.2013.2249063

Cited by 120 publications

(75 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…pressures, facial expressions of emotion comprise specific facial movements [4][5][6][7][8] to support a near-optimal system of signaling and decoding [9,10]. Although highly dynamical [11,12], little is known about the form and function of facial expression temporal dynamics. Do facial expressions transmit diagnostic signals simultaneously to optimize categorization of the six classic emotions, or sequentially to support a more complex communication system of successive categorizations over time?…”

mentioning

confidence: 99%

Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time

2014

View full text Add to dashboard Cite

Designed by biological and social evolutionary pressures, facial expressions of emotion comprise specific facial movements to support a near-optimal system of signaling and decoding. Although highly dynamical, little is known about the form and function of facial expression temporal dynamics. Do facial expressions transmit diagnostic signals simultaneously to optimize categorization of the six classic emotions, or sequentially to support a more complex communication system of successive categorizations over time? Our data support the latter. Using a combination of perceptual expectation modeling, information theory, and Bayesian classifiers, we show that dynamic facial expressions of emotion transmit an evolving hierarchy of "biologically basic to socially specific" information over time. Early in the signaling dynamics, facial expressions systematically transmit few, biologically rooted face signals supporting the categorization of fewer elementary categories (e.g., approach/avoidance). Later transmissions comprise more complex signals that support categorization of a larger number of socially specific categories (i.e., the six classic emotions). Here, we show that dynamic facial expressions of emotion provide a sophisticated signaling system, questioning the widely accepted notion that emotion communication is comprised of six basic (i.e., psychologically irreducible) categories, and instead suggesting four.

show abstract

mentioning

confidence: 99%

Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time

2014

View full text Add to dashboard Cite

show abstract

“…There are many good features that can be extracted from each video to capture the movement of the fingers. Here LBP-TOP [20] and LPQTOP [6] are selected. These features can not only calculate the distribution of the local information of each frame, but also the distribution of finger movements along to the time.…”

Section: Video Based Micro-gesture Recognitionmentioning

confidence: 99%

Holoscopic 3D Micro-Gesture Database for Wearable Device Interaction

Liu¹,

Meng²,

Swash³

et al. 2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

View full text Add to dashboard Cite

With the rapid development of augmented reality (AR) and virtual reality (VR) technology, human-computer interaction (HCI) has been greatly improved for gaming interaction of AR and VR control. The finger micro-gesture is one of the important interactive methods for HCI applications such as in the Google Soli and Microsoft Kinect projects. However, the progress in this research is slow due to the lack of high quality public available database. In this paper, holoscopic 3D camera is used to capture high quality micro-gesture images and a new unique holoscopic 3D micro-gesture (HoMG) database is produced. The principle of the holoscopic 3D camera is based on the flys viewing system to see the objects. HoMG database recorded the image sequence of 3 conventional gestures from 40 participants under different settings and conditions. For the purpose of micro-gesture recognition, HoMG has a video subset with 960 videos and a still image subset with 30635 images. Initial micro-gesture recognition on both subsets has been conducted using traditional 2D image and video features and popular classifiers and some encouraging performance has been achieved. The database will be available for the research communities and speed up the research in this area.

show abstract

“…Methods that do so attempt using either temporal image features [31,32] or DBN-based models such as HMMs [33] and CRFs [34]. In general, these works perform either majority voting using the static detection [25], or detection of the temporal phases of AUs followed by the rulebased classification of the sequences (by detecting the onsetapex-offset sequence of an AU) [33,35]. Other temporal models are based on Ordinal CRFs have been proposed for modeling of AU temporal phases [36], and their intensity [1], however, they do not perform AU detection.…”

Section: Facial Au Detectionmentioning

confidence: 99%

Variable-state Latent Conditional Random Field models for facial expression analysis

Walecki

Rudovic

Pavlović

et al. 2017

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

Automated recognition of facial expressions of emotions, and detection of facial action units (AUs), from videos depends critically on modeling of their dynamics. These dynamics are characterized by changes in temporal phases (onset-apex-offset) and intensity of emotion expressions and AUs, the appearance of which may vary considerably among target subjects, making the recognition/detection task very challenging. The state-of-the-art Latent Conditional Random Fields (L-CRF) framework allows one to efficiently encode these dynamics through the latent states accounting for the temporal consistency in emotion expression and ordinal relationships between its intensity levels, these latent states are typically assumed to be either unordered (nominal) or fully ordered (ordinal). Yet, such an approach is often too restrictive. For instance, in the case of AU detection, the goal is to discriminate between the segments of an image sequence in which this AU is active or inactive. While the sequence segments containing activation of the target AU may better be described using ordinal latent states (corresponding to the AU intensity levels), the inactive segments (i.e., where this AU does not occur) may better be described using unordered (nominal) latent states, as no assumption can be made about their underlying structure (since they can contain either neutral faces or activations of non-target AUs). To address this, we propose the variable-state L-CRF (VSL-CRF) model that automatically selects the optimal latent states for the target image sequence, based on the input data and underlying dynamics of the sequence. To reduce the model overfitting either the nominal or ordinal latent states, we propose a novel graph-Laplacian regularization of the latent states. We evaluate the VSL-CRF on the tasks of facial expression recognition using the CK+ dataset, and AU detection using the GEMEP-FERA and DISFA datasets, and show that the proposed model achieves better generalization performance compared to traditional L-CRFs and other related state-of-the-art models.

show abstract

A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modeling

Cited by 120 publications

References 47 publications

Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time

Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time

Holoscopic 3D Micro-Gesture Database for Wearable Device Interaction

Variable-state Latent Conditional Random Field models for facial expression analysis

Contact Info

Product

Resources

About