Py-Feat: Python Facial Expression Analysis Toolbox

Jolly, Eshin; Cheong, Jin Hyun; Xie, Tiankang; Byrne, Sophie; Kenny, Matthew; Chang, Luke J.

doi:10.48550/arxiv.2104.03509

Cited by 16 publications

(15 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this research, two recently introduced frameworks, i.e. Light-Face [17] and Py-Feat [18] are used as examples of emotion categorization tools to categorize images.…”

Section: Emotion Categorization Toolsmentioning

confidence: 99%

“…Py-Feat is an open-source facial analysis toolbox, including emotion categorization [18]. The framework was trained on three emotion categorization datasets, i.e.…”

Section: Py-featmentioning

confidence: 99%

“…3. Finally, we will investigate the effectiveness of the method compared with seven state-of-the-art methods, including leading DL methods such as the residual neural network (ResNet) [14], the visual geometry group (VGG) [15], the Inception-ResNet [16]), emotion categorization tools such as LightFace [17] and the Python Facial Expression Analysis Toolbox (Py-Feat) [18], as well as other anti-attack methods such as the Deep Locality Preserving Convolution Neural Network (DLP-CNN) [19] and the Adversarially Trained Network (Adv-Network) [20].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An anti-attack method for emotion categorization from images

Shehu

Browne

Eisenbarth

2022

Applied Soft Computing

View full text Add to dashboard Cite

“…In this research, two recently introduced frameworks, i.e. Light-Face [17] and Py-Feat [18] are used as examples of emotion categorization tools to categorize images.…”

Section: Emotion Categorization Toolsmentioning

confidence: 99%

“…Py-Feat is an open-source facial analysis toolbox, including emotion categorization [18]. The framework was trained on three emotion categorization datasets, i.e.…”

Section: Py-featmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An anti-attack method for emotion categorization from images

Shehu

Browne

Eisenbarth

2022

Applied Soft Computing

View full text Add to dashboard Cite

“…This may reflect noise that is a fit to the individual's face morphology rather than to facial expressions of emotion. It should be noted that the assessment of facial movements is largely dependent on the target stimuli and their nature [84], but the state-of-the-art AU detection system comparisons provided average F1 scores of .56-.59 [85]. Perusquia-Herna ´ndez et al [46] also indicate the existence of entanglement between upper lip raising (AU10) and lip corner pulling (AU12).…”

Section: Plos Onementioning

confidence: 99%

The spatio-temporal features of perceived-as-genuine and deliberate expressions

2022

View full text Add to dashboard Cite

Reading the genuineness of facial expressions is important for increasing the credibility of information conveyed by faces. However, it remains unclear which spatio-temporal characteristics of facial movements serve as critical cues to the perceived genuineness of facial expressions. This study focused on observable spatio-temporal differences between perceived-as-genuine and deliberate expressions of happiness and anger expressions. In this experiment, 89 Japanese participants were asked to judge the perceived genuineness of faces in videos showing happiness or anger expressions. To identify diagnostic facial cues to the perceived genuineness of the facial expressions, we analyzed a total of 128 face videos using an automated facial action detection system; thereby, moment-to-moment activations in facial action units were annotated, and nonnegative matrix factorization extracted sparse and meaningful components from all action units data. The results showed that genuineness judgments reduced when more spatial patterns were observed in facial expressions. As for the temporal features, the perceived-as-deliberate expressions of happiness generally had faster onsets to the peak than the perceived-as-genuine expressions of happiness. Moreover, opening the mouth negatively contributed to the perceived-as-genuine expressions, irrespective of the type of facial expressions. These findings provide the first evidence for dynamic facial cues to the perceived genuineness of happiness and anger expressions.

show abstract

“…All target images in the database are labeled based on six basic emotions [13] by human experts, as well as the 20 AUs (Table I ) automatically using a pre-trained classifier provided by Py-Feat [40]. The classifier's inputs are the following two vectors: the facial landmarks, a (68×2) vector of the landmark locations that is computed with the dlib package [41], and the HOGs; a vector of (5408 × 1) features that describe an image as a distribution of orientations [21].…”

Section: A Facegamementioning

confidence: 99%

Interpretable Explainability in Facial Emotion Recognition and Gamification for Data Collection

Shingjergji

İren

Böttger

et al. 2022

2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

Training facial emotion recognition models requires large sets of data and costly annotation processes. To alleviate this problem, we developed a gamified method of acquiring annotated facial emotion data without an explicit labeling effort by humans. The game, which we named Facegame, challenges the players to imitate a displayed image of a face that portrays a particular basic emotion. Every round played by the player creates new data that consists of a set of facial features and landmarks, already annotated with the emotion label of the target facial expression. Such an approach effectively creates a robust, sustainable, and continuous machine learning training process. We evaluated Facegame with an experiment that revealed several contributions to the field of affective computing. First, the gamified data collection approach allowed us to access a rich variation of facial expressions of each basic emotion due to the natural variations in the players' facial expressions and their expressive abilities. We report improved accuracy when the collected data were used to enrich well-known in-the-wild facial emotion datasets and consecutively used for training facial emotion recognition models. Second, the natural language prescription method used by the Facegame constitutes a novel approach for interpretable explainability that can be applied to any facial emotion recognition model. Finally, we observed significant improvements in the facial emotion perception and expression skills of the players through repeated game play.

show abstract

Py-Feat: Python Facial Expression Analysis Toolbox

Cited by 16 publications

References 0 publications

An anti-attack method for emotion categorization from images

An anti-attack method for emotion categorization from images

The spatio-temporal features of perceived-as-genuine and deliberate expressions

Interpretable Explainability in Facial Emotion Recognition and Gamification for Data Collection

Contact Info

Product

Resources

About