Probabilistic Attribute Tree Structured Convolutional Neural Networks for Facial Expression Recognition in the Wild

Cai, Jie; Meng, Zibo; Khan, Arshad; Li, Zhiyuan; O’Reilly, James T.; Tong, Yan

doi:10.1109/taffc.2022.3156920

Cited by 20 publications

(6 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SCN [ 61 ] was proposed to solve the noise label problem. DAS [ 12 ] and PAT [ 59 ] introduce more manual labels into the training data. Compared with algorithms that introduce more human labels into the training, such as the DAS and PAT algorithms, our algorithm achieves better results.…”

Section: Methodsmentioning

confidence: 99%

“…FSN [57] AlexNet 81.10 MRE-CNN [58] VGG-16 82.63 gACNN [24] VGG-16 85.07 PAT-VGG-F-(gender,race) [59] VGG-16 83.83 PAT-ResNet-(gender,race) [59] ResNet-34 84.19 OADN [60] ResNet-50 87.16 SCN [61] ResNet-18 87.03 RAN [23] ResNet-18 86.90 PASM(3 rounds) [23] ResNet-34 88.68 WS-LGRN [62] DesNet 85.79 DAS [12] ResNet-34 85.24 EfficientFace [29] ShuffleNet-V2 88.36 MobileNet-V2(baseline) [63] MobileNet-V2 83.96…”

Section: Methods Backbone Accuracy (%)mentioning

confidence: 99%

“…Inception [41] InceptionNet 93.20 PAT-VGG-F-(gender,race) [59] VGG-16 95.58 PAT-ResNet-(gender,race) [59] ResNet-34 95.82 DAS [12] VGG-16 95.35 WS-LGRN [62] DenseNet 98.37 MobileNet-V2(baseline) [63] MobileNet-V2 94.24 ACNN-EC MobileNet-V2 98.95 Table 6. Inner-dataset experimental results on the self-collected dataset.…”

Section: Methods Backbone Accuracy (%)mentioning

confidence: 99%

“…Compared with some FER algorithms based on an efficient CNN structure, the proposed ACNN-EC also has a higher recognition accuracy. [59] ResNet-34 72.00 PASM(2 rounds) [23] ResNet-34 73.59 DAS [12] ResNet-34 71.23 MobileNet-V2(baseline) [63] MobileNet-V2 72.17…”

Section: 98mentioning

confidence: 99%

See 3 more Smart Citations

Expression-Guided Deep Joint Learning for Facial Expression Recognition

Fang

Zhao

Han

et al. 2023

Sensors

View full text Add to dashboard Cite

In recent years, convolutional neural networks (CNNs) have played a dominant role in facial expression recognition. While CNN-based methods have achieved remarkable success, they are notorious for having an excessive number of parameters, and they rely on a large amount of manually annotated data. To address this challenge, we expand the number of training samples by learning expressions from a face recognition dataset to reduce the impact of a small number of samples on the network training. In the proposed deep joint learning framework, the deep features of the face recognition dataset are clustered, and simultaneously, the parameters of an efficient CNN are learned, thereby marking the data for network training automatically and efficiently. Specifically, first, we develop a new efficient CNN based on the proposed affinity convolution module with much lower computational overhead for deep feature learning and expression classification. Then, we develop an expression-guided deep facial clustering approach to cluster the deep features and generate abundant expression labels from the face recognition dataset. Finally, the AC-based CNN is fine-tuned using an updated training set and a combined loss function. Our framework is evaluated on several challenging facial expression recognition datasets as well as a self-collected dataset. In the context of facial expression recognition applied to the field of education, our proposed method achieved an impressive accuracy of 95.87% on the self-collected dataset, surpassing other existing methods.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methods Backbone Accuracy (%)mentioning

confidence: 99%

Section: Methods Backbone Accuracy (%)mentioning

confidence: 99%

Section: 98mentioning

confidence: 99%

See 2 more Smart Citations

Expression-Guided Deep Joint Learning for Facial Expression Recognition

Fang

Zhao

Han

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…With the improvement of hardware performance and data volume, convolution neural networks have been widely used in facial expression recognition tasks. In many studies [9–11] multi‐task and multi‐scale information was integrated on the basis of classical networks, such as VGG (Visual Geometry Group) [12] and ResNet (Residual Network) [13], so as to improve the recognition accuracy. Qin [14] and Pramerdorfer et al.…”

Section: Introductionmentioning

confidence: 99%

Real‐time facial expression recognition based on iterative transfer learning and efficient attention network

Kong

Zhang

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

Real-time facial expression recognition is the basis for computers to understand human emotions and detect abnormalities in time. To effectively solve the problems of server overload and privacy information leakage, a real-time facial expression recognition method based on iterative transfer learning and efficient attention network (EAN) for edge resource-constrained scenes is proposed in this paper. Firstly, an EAN is designed with its parameter number and computation amount strictly limited by depth separable convolution and local channel attention mechanism. Then, the soft labels of facial expression data were obtained by EAN based on the idea of knowledge distillation, so as to provide more supervision information for the training process. Finally, an iterative transfer learning method of teacher-student (T-S) network was proposed; it refines the soft labels of the teacher network and further improves the recognition accuracy of the student network.The tests on the public datasets, FER2013 and RAF-DB, show that this method can significantly reduce the model complexity and achieve high recognition accuracy. Compared with other advanced methods, the proposed method strikes a good balance between complexity and accuracy, and well meets the real-time deployment requirements of facial expression recognition technology for edge resource-constrained scenes.

show abstract

Harmonious Mutual Learning for Facial Emotion Recognition

Gan,

Xu,

Xia

et al. 2024

Neural Process Lett

View full text Add to dashboard Cite

Facial emotion recognition in the wild is an important task in computer vision, but it still remains challenging since the influence of backgrounds, occlusions and illumination variations in facial images, as well as the ambiguity of expressions. This paper proposes a harmonious mutual learning framework for emotion recognition, mainly through utilizing attention mechanisms and probability distributions without utilizing additional information. Specifically, this paper builds an architecture with two emotion recognition networks and makes progressive cooperation and interaction between them. We first integrate self-mutual attention module into the backbone to learn discriminative features against the influence from emotion-irrelevant facial information. In this process, we deploy spatial attention module and convolutional block attention module for the two networks respectively, guiding to enhanced and supplementary learning of attention. Further, in the classification head, we propose to learn the latent ground-truth emotion probability distributions using softmax function with temperature to characterize the expression ambiguity. On this basis, a probability distribution distillation learning module is constructed to perform class semantic interaction using bi-directional KL loss, allowing mutual calibration for the two networks. Experimental results on three public datasets show the superiority of the proposed method compared to state-of-the-art ones.

show abstract

Probabilistic Attribute Tree Structured Convolutional Neural Networks for Facial Expression Recognition in the Wild

Cited by 20 publications

References 82 publications

Expression-Guided Deep Joint Learning for Facial Expression Recognition

Expression-Guided Deep Joint Learning for Facial Expression Recognition

Real‐time facial expression recognition based on iterative transfer learning and efficient attention network

Harmonious Mutual Learning for Facial Emotion Recognition

Contact Info

Product

Resources

About