Automatic 3D facial expression recognition using geometric and textured feature fusion

Jan, Asim; Meng, Hongying

doi:10.1109/fg.2015.7284860

Cited by 17 publications

(11 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It contains the combination of facial muscles for each expression [12], and can be used as a tool to detect the emotional state of a person through their face. Another approach to classify emotion through facial expressions is using local and holistic feature descriptors, such as in [13]. Unlike FACS, these techniques treat the whole face the same and look for patterns throughout, and not just for certain muscles.…”

mentioning

confidence: 99%

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Jan

Meng

Gaus

et al. 2018

IEEE Trans. Cogn. Dev. Syst.

Self Cite

158

View full text Add to dashboard Cite

A human being's cognitive system can be simulated by artificial intelligent systems. Machines and robots equipped with cognitive capability can automatically recognize a humans mental state through their gestures and facial expressions. In this paper, an artificial intelligent system is proposed to monitor depression. It can predict the scales of Beck depression inventory II (BDI-II) from vocal and visual expressions. First, different visual features are extracted from facial expression images. Deep learning method is utilized to extract key visual features from the facial expression frames. Second, spectral lowlevel descriptors and mel-frequency cepstral coefficients features are extracted from short audio segments to capture the vocal expressions. Third, feature dynamic history histogram (FDHH) is proposed to capture the temporal movement on the feature space. Finally, these FDHH and audio features are fused using regression techniques for the prediction of the BDI-II scales. The proposed method has been tested on the public Audio/Visual Emotion Challenges 2014 dataset as it is tuned to be more focused on the study of depression. The results outperform all the other existing methods on the same dataset.

show abstract

mentioning

confidence: 99%

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Jan

Meng

Gaus

et al. 2018

IEEE Trans. Cogn. Dev. Syst.

Self Cite

158

View full text Add to dashboard Cite

show abstract

“…Features are usually calculated on the region surrounding principal facial landmarks or on the mouth and eyes that inherently contain essential information for emotion recognition. These key features that are considered closely related to expression categories, in order to perform FER are fed to various classifiers, as well as Support-Vector Machines (SVM) [47][48][49][50][51], Adaboost, k-Nearest Neighbors (k-NN), Linear Discriminant Analysis (LDA), Modified Principal Component Analysis (PCA), Hidden Markov Model (HMM) [44][45][46], Random Forest [52] or Neural Networks [51,53,54].…”

Section: Feature-based Vs Model-based Algorithmsmentioning

confidence: 99%

“…Hence, a feature dimensionality reduction technique is used to reduce the size of the feature vector while retaining its quality. Jan and Meng [49] in 2015, proposed to fuse the key features obtained from the geometric and textured domains, to investigate how the overall performance is affected. Also in this work, a feature dimensionality reduction method is used, before applying machine learning techniques; merging the many elements produced by the algorithms can result in a large feature vector which can slow down the system.…”

Section: Multi-modal Algorithms Using 2d and 3d Datamentioning

confidence: 99%

3D Approaches and Challenges in Facial Expression Recognition Algorithms—A Literature Review

et al. 2019

View full text Add to dashboard Cite

In recent years, facial expression analysis and recognition (FER) have emerged as an active research topic with applications in several different areas, including the human-computer interaction domain. Solutions based on 2D models are not entirely satisfactory for real-world applications, as they present some problems of pose variations and illumination related to the nature of the data. Thanks to technological development, 3D facial data, both still images and video sequences, have become increasingly used to improve the accuracy of FER systems. Despite the advance in 3D algorithms, these solutions still have some drawbacks that make pure three-dimensional techniques convenient only for a set of specific applications; a viable solution to overcome such limitations is adopting a multimodal 2D+3D analysis. In this paper, we analyze the limits and strengths of traditional and deep-learning FER techniques, intending to provide the research community an overview of the results obtained looking to the next future. Furthermore, we describe in detail the most used databases to address the problem of facial expressions and emotions, highlighting the results obtained by the various authors. The different techniques used are compared, and some conclusions are drawn concerning the best recognition rates achieved.

show abstract

“…3D FER has become an extensive field of research with many early attempts in [3], [4], [5], [6], [7] and most recent works in [8], [9], [10] that trend to use both 2D and 3D multi-modal data to further improve the accuracy. Huynh et al [11] proposed to use deep CNNs for classifying the six basic facial expressions.…”

Section: Related Workmentioning

confidence: 99%

Accurate Facial Parts Localization and Deep Learning for 3D Facial Expression Recognition

Jan

Ding

Meng

et al. 2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

Self Cite

View full text Add to dashboard Cite

Meaningful facial parts can convey key cues for both facial action unit detection and expression prediction. Textured 3D face scan can provide both detailed 3D geometric shape and 2D texture appearance cues of the face which are beneficial for Facial Expression Recognition (FER). However, accurate facial parts extraction as well as their fusion are challenging tasks. In this paper, a novel system for 3D FER is designed based on accurate facial parts extraction and deep feature fusion of facial parts. In particular, each textured 3D face scan is firstly represented as a 2D texture map and a depth map with one-to-one dense correspondence. Then, the facial parts of both texture map and depth map are extracted using a novel 4-stage process consists of facial landmark localization, facial rotation correction, facial resizing, facial parts bounding box extraction and post-processing procedures. Finally, deep fusion Convolutional Neural Networks (CNNs) features of all facial parts are learned from both texture maps and depth maps, respectively and nonlinear SVMs are used for expression prediction. Experiments are conducted on the BU-3DFE database, demonstrating the effectiveness of combing different facial parts, texture and depth cues and reporting the state-of-the-art results in comparison with all existing methods under the same setting.

show abstract

Automatic 3D facial expression recognition using geometric and textured feature fusion

Cited by 17 publications

References 22 publications

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

3D Approaches and Challenges in Facial Expression Recognition Algorithms—A Literature Review

Accurate Facial Parts Localization and Deep Learning for 3D Facial Expression Recognition

Contact Info

Product

Resources

About