OpenFace: An open source facial behavior analysis toolkit

Baltrušaitis, Tadas; Robinson, Peter; Morency, Louis–Philippe

doi:10.1109/wacv.2016.7477553

Cited by 1,048 publications

(701 citation statements)

References 53 publications

Supporting

Mentioning

697

Contrasting

Unclassified

Order By: Relevance

“…(1). Lastly, we extract facial features of each frame with the OpenFace toolkit [34]. We extract 2D position of the facial landmarks, as well as Action Unit (AU) intensities, and treat them as two separate feature sets.…”

Section: Set-upmentioning

confidence: 99%

Construction of Spontaneous Emotion Corpus from Indonesian TV Talk Shows and Its Application on Multimodal Emotion Recognition

Lubis

Lestari

Sakti

et al. 2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARY As interaction between human and computer continues to develop to the most natural form possible, it becomes increasingly urgent to incorporate emotion in the equation. This paper describes a step toward extending the research on emotion recognition to Indonesian. The field continues to develop, yet exploration of the subject in Indonesian is still lacking. In particular, this paper highlights two contributions: (1) the construction of the first emotional audio-visual database in Indonesian, and (2) the first multimodal emotion recognizer in Indonesian, built from the aforementioned corpus. In constructing the corpus, we aim at natural emotions that are corresponding to real life occurrences. However, the collection of emotional corpora is notably labor intensive and expensive. To diminish the cost, we collect the emotional data from television programs recordings, eliminating the need of an elaborate recording set up and experienced participants. In particular, we choose television talk shows due to its natural conversational content, yielding spontaneous emotion occurrences. To cover a broad range of emotions, we collected three episodes in different genres: politics, humanity, and entertainment. In this paper, we report points of analysis of the data and annotations. The acquisition of the emotion corpus serves as a foundation in further research on emotion. Subsequently, in the experiment, we employ the support vector machine (SVM) algorithm to model the emotions in the collected data. We perform multimodal emotion recognition utilizing the predictions of three modalities: acoustic, semantic, and visual. When compared to the unimodal result, in the multimodal feature combination, we attain identical accuracy for the arousal at 92.6%, and a significant improvement for the valence classification task at 93.8%. We hope to continue this work and move towards a finer-grain, more precise quantification of emotion.

show abstract

Section: Set-upmentioning

confidence: 99%

Construction of Spontaneous Emotion Corpus from Indonesian TV Talk Shows and Its Application on Multimodal Emotion Recognition

Lubis

Lestari

Sakti

et al. 2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…1(a). To this end, a state-of-the-art tracker [2] is used. The tracker uses an extended version of Conditional Local Neural Fields (CLNF) [1], where individual point distribution and patch expert models are learned for eyes, lips and eyebrows.…”

Section: Facial Landmark Tracking and Alignmentmentioning

confidence: 99%

Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification

Dibeklioğlu

2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…We use Conditional with Local Neural Fields (CLNF) with OpenFace toolkit [35] to detect the face locate the key points on the face (show in Figure 3a). Every frame is aligned and cropped according to the key points.…”

Section: Data Preprocessingmentioning

confidence: 99%

NIRExpNet: Three-Stream 3D Convolutional Neural Network for Near Infrared Facial Expression Recognition

Chen

et al. 2017

Applied Sciences

View full text Add to dashboard Cite

Facial expression recognition (FER) under active near-infrared (NIR) illumination has the advantages of illumination invariance. In this paper, we propose a three-stream 3D convolutional neural network, named as NIRExpNet for NIR FER. The 3D structure of NIRExpNet makes it possible to extract automatically, not just spatial features, but also, temporal features. The design of multiple streams of the NIRExpNet enables it to fuse local and global facial expression features. To avoid over-fitting, the NIRExpNet has a moderate size to suit the Oulu-CASIA NIR facial expression database that is a medium-size database. Experimental results show that the proposed NIRExpNet outperforms some previous state-of-art methods, such as Histogram of Oriented Gradient to 3D (HOG 3D), Local binary patterns from three orthogonal planes (LBP-TOP), deep temporal appearance-geometry network (DTAGN), and adapt 3D Convolutional Neural Networks (3D CNN DAP).

show abstract

OpenFace: An open source facial behavior analysis toolkit

Cited by 1,048 publications

References 53 publications

Construction of Spontaneous Emotion Corpus from Indonesian TV Talk Shows and Its Application on Multimodal Emotion Recognition

Construction of Spontaneous Emotion Corpus from Indonesian TV Talk Shows and Its Application on Multimodal Emotion Recognition

Visual Transformation Aided Contrastive Learning for Video-Based Kinship Verification

NIRExpNet: Three-Stream 3D Convolutional Neural Network for Near Infrared Facial Expression Recognition

Contact Info

Product

Resources

About