Facial Action Units for Training Convolutional Neural Networks

Pham, Trinh Thi Doan; Won, Chee Sun

doi:10.1109/access.2019.2921241

Cited by 15 publications

(2 citation statements)

References 38 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the fact that in our database, we used Caucasian subjects exclusively, this did not pose a concern for further tests conducted to evaluate DASS levels. We compared the average accuracy obtained by using this multiclass SVM-based method to other state-of-the-art CNN-based methods for AU classification (CNN-based classifiers [63], multilabel CNN classifiers [64], and a combination of CNN-based and SVM-based classifiers [65]) that are known to offer high classification accuracy, and the results are displayed in Figure 3. As can be observed, the proposed multiclass SVM-based method offered an accuracy comparable to the one obtained using the method with CNN-based and SVM-based classifiers proposed in Reference [65], but it was significantly faster (hence why we used it in this study, as we aimed to build a system able to predict DASS levels in real time).…”

Section: Methodsmentioning

confidence: 99%

Predicting Depression, Anxiety, and Stress Levels from Videos Using the Facial Action Coding System

Gavrilescu

Vizireanu

2019

Sensors

104

View full text Add to dashboard Cite

We present the first study in the literature that has aimed to determine Depression Anxiety Stress Scale (DASS) levels by analyzing facial expressions using Facial Action Coding System (FACS) by means of a unique noninvasive architecture on three layers designed to offer high accuracy and fast convergence: in the first layer, Active Appearance Models (AAM) and a set of multiclass Support Vector Machines (SVM) are used for Action Unit (AU) classification; in the second layer, a matrix is built containing the AUs’ intensity levels; and in the third layer, an optimal feedforward neural network (FFNN) analyzes the matrix from the second layer in a pattern recognition task, predicting the DASS levels. We obtained 87.2% accuracy for depression, 77.9% for anxiety, and 90.2% for stress. The average prediction time was 64 s, and the architecture could be used in real time, allowing health practitioners to evaluate the evolution of DASS levels over time. The architecture could discriminate with 93% accuracy between healthy subjects and those affected by Major Depressive Disorder (MDD) or Post-traumatic Stress Disorder (PTSD), and 85% for Generalized Anxiety Disorder (GAD). For the first time in the literature, we determined a set of correlations between DASS, induced emotions, and FACS, which led to an increase in accuracy of 5%. When tested on AVEC 2014 and ANUStressDB, the method offered 5% higher accuracy, sensitivity, and specificity compared to other state-of-the-art methods.

show abstract

Section: Methodsmentioning

confidence: 99%

Predicting Depression, Anxiety, and Stress Levels from Videos Using the Facial Action Coding System

Gavrilescu

Vizireanu

2019

Sensors

104

View full text Add to dashboard Cite

show abstract

“…Deep learning approaches for facial expression approaches use the powerful feature detectors of deep CNN to extract facial expression features, and achieve high performance by deepening the layers of the network and developing effective learning mechanisms. Therefore, they are robust for various face positions and scale changes [11][12][13]. Though these methods are effective and have gained huge success, deep CNN models often carry significant computational cost and require large scale of parameters.…”

Section: Introductionmentioning

confidence: 99%

Efficient Recognition of Facial Expression with Lightweight Octave Convolutional Neural Network

Lai¹,

Chen²,

Li³

et al. 2022

jist

View full text Add to dashboard Cite

In various social activities, people usually focus on the other person’s facial expression, which is an important element in interpersonal communication. The facial expression typically reflects a person’s current mood and conveys emotional information. Perceiving the facial expressions of people in images through cameras has always been a popular research topic. Previous studies have classified facial expressions into different categories, such as happy, sad, fear, angry, calm, disgust and surprise, and have identified them using image processing methods. However, traditional image processing methods have a low detection efficiency and low recognition accuracy due to variation of perspectives. As a result, most of them can only be applied to the front face and short distance situations. In this paper, we propose a lightweight deep learning framework for facial expression recognition using Octave convolutional neural network (FerOctNet). FerOctNet including multi-scale convolutional processing and residual learning is able to obtain multi-scale features with enriched representation ability by integrating the deep level features with rich semantic information with shallow details of the features. Compared with other deep learning networks, not only does the proposed method have a good recognition rate, but also contains fewer parameters in the network.

show abstract

Data fusion analysis for attention‐deficit hyperactivity disorder emotion recognition with thermal image and Internet of Things devices

et al. 2020

View full text Add to dashboard Cite

Summary Attention‐deficit hyperactivity disorder (ADHD) is a symptom of behavioral or emotional problems as these problems affect children's learning and social integration. With the advancements in the Internet of Things (IoTs), emotions can be detected through image and physiological data. However, some critical ADHD children are often accompanied by the inability to control their body and even facial expressions, making emotion recognition technologies difficult to develop successfully. This study aims to predict the emotions of ADHD children and to address their emotional problems with related IoT robotic devices. Data fusion analysis technology for facial expressions was used to combine thermal images and recognition data, while deep reinforcement learning technology was used to periodically stream information for ADHD students, in alignment with intervention strategies that were designed to address behavioral problems.

show abstract

Facial Action Units for Training Convolutional Neural Networks

Cited by 15 publications

References 38 publications

Predicting Depression, Anxiety, and Stress Levels from Videos Using the Facial Action Coding System

Predicting Depression, Anxiety, and Stress Levels from Videos Using the Facial Action Coding System

Efficient Recognition of Facial Expression with Lightweight Octave Convolutional Neural Network

Data fusion analysis for attention‐deficit hyperactivity disorder emotion recognition with thermal image and Internet of Things devices

Contact Info

Product

Resources

About