Sixteen facial expressions occur in similar contexts worldwide

Cowen, Alan; Keltner, Dacher; Schroff, Florian; Jou, Brendan; Adam, Hartwig; Prasad, Gautam

doi:10.1038/s41586-020-3037-7

Cited by 156 publications

(108 citation statements)

References 52 publications

Supporting

Mentioning

103

Contrasting

Unclassified

Order By: Relevance

“…Although it remains unclear with which database FaceReader was trained, at least two open-source systems (i.e., OpenFace and AFAR) were able to show better performance than chance with three databases that were not trained. Given that one database (Aff-wild2) is comprised of ecologically valid footage derived from YouTube [ 44 ] and the other one (GFT) comprised social conversation recorded in a natural environment, these results indicate the generalizability of the extant automatic AU systems.…”

Section: Discussionmentioning

confidence: 99%

Assessing Automated Facial Action Unit Detection Systems for Analyzing Cross-Domain Facial Expression Databases

Namba¹,

Sato²,

Osumi³

et al. 2021

Sensors

View full text Add to dashboard Cite

In the field of affective computing, achieving accurate automatic detection of facial movements is an important issue, and great progress has already been made. However, a systematic evaluation of systems that now have access to the dynamic facial database remains an unmet need. This study compared the performance of three systems (FaceReader, OpenFace, AFARtoolbox) that detect each facial movement corresponding to an action unit (AU) derived from the Facial Action Coding System. All machines could detect the presence of AUs from the dynamic facial database at a level above chance. Moreover, OpenFace and AFAR provided higher area under the receiver operating characteristic curve values compared to FaceReader. In addition, several confusion biases of facial components (e.g., AU12 and AU14) were observed to be related to each automated AU detection system and the static mode was superior to dynamic mode for analyzing the posed facial database. These findings demonstrate the features of prediction patterns for each system and provide guidance for research on facial expressions.

show abstract

Section: Discussionmentioning

confidence: 99%

Assessing Automated Facial Action Unit Detection Systems for Analyzing Cross-Domain Facial Expression Databases

Namba¹,

Sato²,

Osumi³

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Unlike humans, farm animals are not capable of hiding or deceiving their true emotions, hence the developed system is expected to exert an influence in the welfare of farm animals. Analysis of human facial expressions from 6 million video clips from 144 countries around the world using deep neural network [44] determined that 16 expressions are significantly similar in the way the facial expressions are displayed in varying social contexts. This study further clarified that 70% of human facial expressions as emotional responses are shared across cultures.…”

Section: Discussionmentioning

confidence: 99%

Happy Cow or Thinking Pig? WUR Wolf—Facial Coding Platform for Measuring Emotions in Farm Animals

Neethirajan

2021

View full text Add to dashboard Cite

Emotions play an indicative and informative role in the investigation of farm animal behaviors. Systems that respond and can measure emotions provide a natural user interface in enabling the digitalization of animal welfare platforms. The faces of farm animals can be one of the richest channels for expressing emotions. WUR Wolf (Wageningen University & Research: Wolf Mascot), a real-time facial recognition platform that can automatically code the emotions of farm animals, is presented in this study. The developed Python-based algorithms detect and track the facial features of cows and pigs, analyze the appearance, ear postures, and eye white regions, and correlate these with the mental/emotional states of the farm animals. The system is trained on a dataset of facial features of images of farm animals collected in over six farms and has been optimized to operate with an average accuracy of 85%. From these, the emotional states of animals in real time are determined. The software detects 13 facial actions and an inferred nine emotional states, including whether the animal is aggressive, calm, or neutral. A real-time emotion recognition system based on YoloV3, a Faster YoloV4-based facial detection platform and an ensemble Convolutional Neural Networks (RCNN) is presented. Detecting facial features of farm animals simultaneously in real time enables many new interfaces for automated decision-making tools for livestock farmers. Emotion sensing offers a vast potential for improving animal welfare and animal–human interactions.

show abstract

“…6 The universality hypothesis for facial expression is attributed to Charles Darwin, who thought that at least seven categories of facial expressions-the six BFE and additionally contempt-are recognizable by all humans [15]. The extent to which facial expressions are universal remains a matter of controversy [4,5] and a topic of active research [11,53]. 7 The publicly available version of JAFFE contains 213 images: ten expressers posing three (or in a few cases four) examples of six expressions and a neutral pose.…”

Section: What Is Jaffe?mentioning

confidence: 99%

“…The article [27] was shortlisted for a best paper award and invited to be published in a special issue of the Elsevier journal Image and Vision Computing. 11 After hearing my talk, Zhengyou Zhang, a computer vision researcher who visited ATR in 1997, suggested we extend the visual-cortex-inspired model by training and testing a multi-layer neural network to classify the JAFFE facial expression images. By leveraging existing work, the project progressed quickly, and an article [58] was submitted to the same edition of the IEEE Face and Gesture conference.…”

Section: Jaffe As a Training Setmentioning

confidence: 99%

“…10 The Advanced Telecommunications Research Institute International (国際電気通信基礎技術研究所) an institute located in Kyoto devoted to basic research, where I worked from 1996-2007. 11 The final manuscript that we prepared and submitted for publication in the Image and Vision Computing IEEE FG'98 special issue is available at the arXiv open access repository [39]. The special issue of Image and Vision Computing never materialized.…”

Section: Jaffe As a Training Setmentioning

confidence: 99%

See 1 more Smart Citation

“Excavating AI” Re-excavated: Debunking a Fallacious Account of the JAFFE Dataset

Lyons¹

2021

Preprint

View full text Add to dashboard Cite

Twenty-five years ago, my colleagues Miyuki Kamachi and Jiro Gyoba and I designed and photographed JAFFE, a set of facial expression images intended for use in a study of face perception. In 2019, without seeking permission or informing us, Kate Crawford and Trevor Paglen exhibited JAFFE in two widely publicized art shows. In addition, they published a nonfactual account of the images in the essay “Excavating AI: The Politics of Images in Machine Learning Training Sets.” The present article recounts the creation of the JAFFE dataset and unravels each of Crawford and Paglen’s fallacious statements. I also discuss JAFFE more broadly in connection with research on facial expression, affective computing, and human-computer interaction.

show abstract

Sixteen facial expressions occur in similar contexts worldwide

Cited by 156 publications

References 52 publications

Assessing Automated Facial Action Unit Detection Systems for Analyzing Cross-Domain Facial Expression Databases

Assessing Automated Facial Action Unit Detection Systems for Analyzing Cross-Domain Facial Expression Databases

Happy Cow or Thinking Pig? WUR Wolf—Facial Coding Platform for Measuring Emotions in Farm Animals

“Excavating AI” Re-excavated: Debunking a Fallacious Account of the JAFFE Dataset

Contact Info

Product

Resources

About