Automatic facial coding versus electromyography of mimicked, passive, and inhibited facial response to emotional faces

Höfling, T. Tim A.; Alpers, Georg W.; Gerdes, Antje B. M.; Föhl, Ulrich

doi:10.1080/02699931.2021.1902786

Cited by 20 publications

(22 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although emotion recognition performance is generally lower for such facial expressions, FER tools perform similarly or better than humans for most emotion categories of non-standardized (except for anger and fear) and standardized facial expressions. Facial expressions of joy are detected best among the emotion categories in both standardized and non-standardized facial expressions, which also replicates existing findings (Stöckli et al, 2018 ; Höfling et al, 2021 ). However, FER performance varies strongly between systems and emotion categories.…”

Section: Discussionsupporting

confidence: 89%

“…Höfling et al (2020) report strong correlations of FER parameters and participants' emotion ratings that spontaneously respond to pleasant emotional scenes, but find no evidence for a valid FER detection of spontaneous unpleasant facial reactions. Other studies report a decrease in FER emotion recognition for more subtle and naturalistic facial expressions (Höfling et al, 2021) and find a superiority of humans to decode such emotional facial responses (Yitzhak et al, 2017;Dupré et al, 2020). However, the data sets applied are still comprised of images collected in a controlled lab setting, with little variation on lighting, camera angle, or age of the subject which might further decrease FER performance under less restricted recording conditions.…”

Section: Prototypical Vs Naturalistic Facial Expressionsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions

2021

Self Cite

View full text Add to dashboard Cite

Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition (FER). Based on machine learning, such technology can tremendously increase the amount of processed data. FER is now easily accessible and has been validated for the classification of standardized prototypical facial expressions. However, applicability to more naturalistic facial expressions still remains uncertain. Hence, we test and compare performance of three different FER systems (Azure Face API, Microsoft; Face++, Megvii Technology; FaceReader, Noldus Information Technology) with human emotion recognition (A) for standardized posed facial expressions (from prototypical inventories) and (B) for non-standardized acted facial expressions (extracted from emotional movie scenes). For the standardized images, all three systems classify basic emotions accurately (FaceReader is most accurate) and they are mostly on par with human raters. For the non-standardized stimuli, performance drops remarkably for all three systems, but Azure still performs similarly to humans. In addition, all systems and humans alike tend to misclassify some of the non-standardized emotional facial expressions as neutral. In sum, emotion recognition by automated facial expression recognition can be an attractive alternative to human emotion recognition for standardized and non-standardized emotional facial expressions. However, we also found limitations in accuracy for specific facial expressions; clearly there is need for thorough empirical evaluation to guide future developments in computer vision of emotional facial expressions.

show abstract

Section: Discussionsupporting

confidence: 89%

Section: Prototypical Vs Naturalistic Facial Expressionsmentioning

confidence: 99%

Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…There is evidence that an emotional prototype face is likely to be a better approximation of the centre of emotional 'face space' than a neutral face [57], and a neutral face is not without emotion. For example, neutral and angry faces have been found to elicit comparable negative facial responses when passively viewed, which may indicate that neutral faces are perceived to be negatively valenced [58]. Finally, 15-image linear morph sequences were generated for each emotion, ranging in equally spaced emotional intensities from the emotional prototype (an emotionally ambiguous face; 5% emotional signal) to the emotional exemplar (the full intensity unambiguous emotion; 100% emotional signal).…”

Section: Six-alternative Forced Choice (6afc) Taskmentioning

confidence: 99%

The role of state and trait anxiety in the processing of facial expressions of emotion

et al. 2022

View full text Add to dashboard Cite

State anxiety appears to influence facial emotion processing (Attwood et al . 2017 R. Soc. Open Sci. 4 , 160855). We aimed to (i) replicate these findings and (ii) investigate the role of trait anxiety, in an experiment with healthy UK participants ( N = 48, 50% male, 50% high trait anxiety). High and low state anxiety were induced via inhalations of 7.5% carbon dioxide enriched air and medical air, respectively. High state anxiety reduced global emotion recognition accuracy ( p = 0.01, η p 2 = 0.14 ), but it did not affect interpretation bias towards perceiving anger in ambiguous angry–happy facial morphs ( p = 0.18, η p 2 = 0.04 ). We found no clear evidence of a relationship between trait anxiety and global emotion recognition accuracy ( p = 0.60, η p 2 = 0.01 ) or interpretation bias towards perceiving anger ( p = 0.83, η p 2 = 0.01 ). However, there was greater interpretation bias towards perceiving anger (i.e. away from happiness) during heightened state anxiety, among individuals with high trait anxiety ( p = 0.03, d z = 0.33). State anxiety appears to impair emotion recognition accuracy, and among individuals with high trait anxiety, it appears to increase biases towards perceiving anger (away from happiness). Trait anxiety alone does not appear to be associated with facial emotion processing.

show abstract

“…In comparison to human FACS coding, automatic facial coding (AFC) offers several advantages: it is dramatically more time efficient because it can analyze a large number of facial expressions without human effort [ 10 ]. Moreover, AFC is less intrusive and less susceptible to motion artifacts [ 11 ], but also less sensitive to more subtle facial responses compared to psycho-physiological measures like electromyography [ 12 , 13 ].…”

Section: Introductionmentioning

confidence: 99%

“…Only few studies have also tested the validity of AFC in more naturalistic facial expressions of untrained participants who posed facial expressions. Two studies documented that AFC is sensitive for posed joy and anger, but with larger sensitivity for joyful compared to angry faces [ 13 , 18 ]. Two other studies, in which participants posed all six emotions, reported substantial differences in sensitivity for specific emotion categories [ 21 , 25 ].…”

Section: Introductionmentioning

confidence: 99%

What’s in a face: Automatic facial coding of untrained study participants compared to standardized inventories

et al. 2022

Self Cite

View full text Add to dashboard Cite

Automatic facial coding (AFC) is a novel research tool to automatically analyze emotional facial expressions. AFC can classify emotional expressions with high accuracy in standardized picture inventories of intensively posed and prototypical expressions. However, classification of facial expressions of untrained study participants is more error prone. This discrepancy requires a direct comparison between these two sources of facial expressions. To this end, 70 untrained participants were asked to express joy, anger, surprise, sadness, disgust, and fear in a typical laboratory setting. Recorded videos were scored with a well-established AFC software (FaceReader, Noldus Information Technology). These were compared with AFC measures of standardized pictures from 70 trained actors (i.e., standardized inventories). We report the probability estimates of specific emotion categories and, in addition, Action Unit (AU) profiles for each emotion. Based on this, we used a novel machine learning approach to determine the relevant AUs for each emotion, separately for both datasets. First, misclassification was more frequent for some emotions of untrained participants. Second, AU intensities were generally lower in pictures of untrained participants compared to standardized pictures for all emotions. Third, although profiles of relevant AU overlapped substantially across the two data sets, there were also substantial differences in their AU profiles. This research provides evidence that the application of AFC is not limited to standardized facial expression inventories but can also be used to code facial expressions of untrained participants in a typical laboratory setting.

show abstract

Automatic facial coding versus electromyography of mimicked, passive, and inhibited facial response to emotional faces

Cited by 20 publications

References 81 publications

Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions

Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions

The role of state and trait anxiety in the processing of facial expressions of emotion

What’s in a face: Automatic facial coding of untrained study participants compared to standardized inventories

Contact Info

Product

Resources

About