Refactoring facial expressions: An automatic analysis of natural occurring facial expressions in iterative social dilemma

Stratou, Giota; Schalk, Job van der; Hoegen, Rens; Gratch, Jonathan

doi:10.1109/acii.2017.8273635

Cited by 19 publications

(20 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, results of this study hint towards the contrary; the most frequent facial displays are neutral or have only a single activated AU instead of a combination of AUs. Further research is needed to compare the approach of Stratou et al [40] with the current study.…”

Section: Discussionmentioning

confidence: 95%

“…in an emotional environment), the precedence should be given to that particular signal. A recent work by Stratou et al has reduced the AU space by using factors that describe combinations of correlated AU activations [40] ; such approach implies that ECA facial displays can be manipulated with a small number of factors and a change in a specific factor would signify a change in all AUs related to that factor. However, results of this study hint towards the contrary; the most frequent facial displays are neutral or have only a single activated AU instead of a combination of AUs.…”

Section: Discussionmentioning

confidence: 99%

“…However, the current research does analyse both combinations and single AU activations, therefore it is essential that each AU is accurately encoded. For that reason, in the current work we only selected manually encoded datasets with high interrater reliability scores (in contrast to other approaches like [40]). The number of available human FACS-encoded datasets containing non-posed facial configurations is rather small [21].…”

Section: Current Workmentioning

confidence: 99%

See 2 more Smart Citations

Spontaneous Facial Behavior Revolves Around Neutral Facial Displays

Blomsma

Vaitonytė

Alimardani

et al. 2020

Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

View full text Add to dashboard Cite

With forty-six Action Units (AUs) forming the building blocks in the Facial Action Coding System (FACS), millions of facial configurations can be formed. Most research has focused on a subset of combinations to determine the link between facial configurations and emotions. Despite the value of this research for psychological and computational reasons, it is not clear what the most common combinations of AUs are to form the most commonly expressed facial configurations. We used three diverse corpora with human coded facial action units for a computational analysis. The analysis demonstrated that the largest portion of facial behavior consists of the absence of AU activations, yielding only one specific facial configuration, that of the neutral face. These results are important for cognitive scientists, computer graphics designers and virtual human developers alike. They suggest that only a relatively small number of AU combinations are initially needed for the creation of natural facial behavior in Embodied Conversational Agents (ECAs).

show abstract

Section: Discussionmentioning

confidence: 95%

Section: Discussionmentioning

confidence: 99%

Section: Current Workmentioning

confidence: 99%

See 1 more Smart Citation

Spontaneous Facial Behavior Revolves Around Neutral Facial Displays

Blomsma

Vaitonytė

Alimardani

et al. 2020

Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

View full text Add to dashboard Cite

show abstract

“…This is likely due to the relatively coarse level of detail that the facial action coding models provide, compared to the fine-scale pixel-level detail in facial texture and shape change offered by the McGM-PCA model. The use of PCA to capture critical dimensions of facial variation from images has previously been employed to describe facial emotions [ 17 , 39 ], but also features such as facial identity [ 23 , 40 , 41 ], gender [ 42 ], and race [ 43 ]. Such approaches typically apply PCA to the original images or after first morphing the faces to a common template to provide a “shape-free” texture representation [ 17 , 41 ].…”

Section: Discussionmentioning

confidence: 99%

A data-driven characterisation of natural facial expressions when giving good and bad news

2020

View full text Add to dashboard Cite

Facial expressions carry key information about an individual’s emotional state. Research into the perception of facial emotions typically employs static images of a small number of artificially posed expressions taken under tightly controlled experimental conditions. However, such approaches risk missing potentially important facial signals and within-person variability in expressions. The extent to which patterns of emotional variance in such images resemble more natural ambient facial expressions remains unclear. Here we advance a novel protocol for eliciting natural expressions from dynamic faces, using a dimension of emotional valence as a test case. Subjects were video recorded while delivering either positive or negative news to camera, but were not instructed to deliberately or artificially pose any specific expressions or actions. A PCA-based active appearance model was used to capture the key dimensions of facial variance across frames. Linear discriminant analysis distinguished facial change determined by the emotional valence of the message, and this also generalised across subjects. By sampling along the discriminant dimension, and back-projecting into the image space, we extracted a behaviourally interpretable dimension of emotional valence. This dimension highlighted changes commonly represented in traditional face stimuli such as variation in the internal features of the face, but also key postural changes that would typically be controlled away such as a dipping versus raising of the head posture from negative to positive valences. These results highlight the importance of natural patterns of facial behaviour in emotional expressions, and demonstrate the efficacy of using data-driven approaches to study the representation of these cues by the perceptual system. The protocol and model described here could be readily extended to other emotional and non-emotional dimensions of facial variance.

show abstract

“…Previous studies proved that combinations of AUs can account for more variation in behaviour than single AUs alone [27]. In addition, several lines of evidence suggest that the combination of AUs predict behaviour more precisely [28].…”

Section: Introductionmentioning

confidence: 99%

Predicting Group Contribution Behaviour in a Public Goods Game from Face-to-Face Communication

Othman

Saxen

Bershadskyy

et al. 2019

Sensors

View full text Add to dashboard Cite

Experimental economic laboratories run many studies to test theoretical predictions with actual human behaviour, including public goods games. With this experiment, participants in a group have the option to invest money in a public account or to keep it. All the invested money is multiplied and then evenly distributed. This structure incentivizes free riding, resulting in contributions to the public goods declining over time. Face-to-face Communication (FFC) diminishes free riding and thus positively affects contribution behaviour, but the question of how has remained mostly unknown. In this paper, we investigate two communication channels, aiming to explain what promotes cooperation and discourages free riding. Firstly, the facial expressions of the group in the 3-minute FFC videos are automatically analysed to predict the group behaviour towards the end of the game. The proposed automatic facial expressions analysis approach uses a new group activity descriptor and utilises random forest classification. Secondly, the contents of FFC are investigated by categorising strategy-relevant topics and using meta-data. The results show that it is possible to predict whether the group will fully contribute to the end of the games based on facial expression data from three minutes of FFC, but deeper understanding requires a larger dataset. Facial expression analysis and content analysis found that FFC and talking until the very end had a significant, positive effect on the contributions.

show abstract

Refactoring facial expressions: An automatic analysis of natural occurring facial expressions in iterative social dilemma

Cited by 19 publications

References 22 publications

Spontaneous Facial Behavior Revolves Around Neutral Facial Displays

Spontaneous Facial Behavior Revolves Around Neutral Facial Displays

A data-driven characterisation of natural facial expressions when giving good and bad news

Predicting Group Contribution Behaviour in a Public Goods Game from Face-to-Face Communication

Contact Info

Product

Resources

About