Island Loss for Learning Discriminative Features in Facial Expression Recognition

Cai, Jie; Meng, Zibo; Khan, Arshad; Li, Zhiyuan; O’Reilly, James T.; Tong, Yan

doi:10.1109/fg.2018.00051

Cited by 261 publications

(152 citation statements)

References 57 publications

Supporting

Mentioning

148

Contrasting

Unclassified

Order By: Relevance

“…CNN+Island loss [24] The last three frames and the first frame appearance-geometry network. Cai et al [24] select the last three frames and the first frame for each video, and train CNN models with a new Island loss function. We argue that manual data selection is an ad-hoc operation on CK+ and it is impractical since we can not know which is the peak frame beforeahead.…”

Section: Methodsmentioning

confidence: 99%

Frame Attention Networks for Facial Expression Recognition in Videos

Meng

Peng

Wang

et al. 2019

2019 IEEE International Conference on Image Processing (ICIP)

162

View full text Add to dashboard Cite

The video-based facial expression recognition aims to classify a given video into several basic emotions. How to integrate facial features of individual frames is crucial for this task. In this paper, we propose the Frame Attention Networks (FAN) 1 , to automatically highlight some discriminative frames in an end-to-end framework. The network takes a video with a variable number of face images as its input and produces a fixed-dimension representation. The whole network is composed of two modules. The feature embedding module is a deep Convolutional Neural Network (CNN) which embeds face images into feature vectors. The frame attention module learns multiple attention weights which are used to adaptively aggregate the feature vectors to form a single discriminative video representation. We conduct extensive experiments on CK+ and AFEW8.0 datasets. Our proposed FAN shows superior performance compared to other CNN based methods and achieves state-of-the-art performance on CK+.

show abstract

Section: Methodsmentioning

confidence: 99%

Frame Attention Networks for Facial Expression Recognition in Videos

Meng

Peng

Wang

et al. 2019

2019 IEEE International Conference on Image Processing (ICIP)

162

View full text Add to dashboard Cite

show abstract

“…22 8 Static Liu et al [41] 94. 19 7 Dynamic Cai et al [42] 94. 35 7 Static Meng et al [43] 95.37 7 static li et al [44] 95.78 6 static chu et al [45] 96.40 7 Dynamic Ding et al [4] 96.8 8 Static Mollahosseini et al [10] 97.80 7 Static Zhao et al [46] 97.…”

Section: Methodsmentioning

confidence: 99%

“…Accuracy # classes D/S Jung et al [18] 74.17 6 Dynamic Kacem et al [20] 83. 13 6 Dynamic Liu et al [41] 74.59 6 Dynamic Guo et al [47] 75.52 6 Dynamic Cai et al [42] 77. 29 6 Static Ding et al [4] 82.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories

Otberdout

Kacem

Daoudi

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

In this paper, we propose a new approach for facial expression recognition using deep covariance descriptors. The solution is based on the idea of encoding local and global Deep Convolutional Neural Network (DCNN) features extracted from still images, in compact local and global covariance descriptors. The space geometry of the covariance matrices is that of Symmetric Positive Definite (SPD) matrices. By conducting the classification of static facial expressions using Support Vector Machine (SVM) with a valid Gaussian kernel on the SPD manifold, we show that deep covariance descriptors are more effective than the standard classification with fully connected layers and softmax. Besides, we propose a completely new and original solution to model the temporal dynamic of facial expressions as deep trajectories on the SPD manifold. As an extension of the classification pipeline of covariance descriptors, we apply SVM with valid positive definite kernels derived from global alignment for deep covariance trajectories classification. By performing extensive experiments on the Oulu-CASIA, CK+, and SFEW datasets, we show that both the proposed static and dynamic approaches achieve state-of-the-art performance for facial expression recognition outperforming many recent approaches.Index Terms-Convolutional neural networks, covariance matrix, deep trajectory, facial expression recognition, symmetric positive definite manifold.

show abstract

“…(b) Island loss layer in [140]. The island loss calculated at the feature extraction layer and the softmax loss calculated at the decision layer are combined to supervise the CNN training.…”

Section: Auxiliary Blocks and Layersmentioning

confidence: 99%

Deep Facial Expression Recognition: A Survey

Deng

2022

IEEE Trans. Affective Comput.

1,082

593

View full text Add to dashboard Cite

With the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose and identity bias. In this paper, we provide a comprehensive survey on deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. We then describe the standard pipeline of a deep FER system with the related background knowledge and suggestions of applicable implementations for each stage. For the state of the art in deep FER, we review existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences, and discuss their advantages and limitations. Competitive performances on widely used benchmarks are also summarized in this section. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.

show abstract

Island Loss for Learning Discriminative Features in Facial Expression Recognition

Cited by 261 publications

References 57 publications

Frame Attention Networks for Facial Expression Recognition in Videos

Frame Attention Networks for Facial Expression Recognition in Videos

Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories

Deep Facial Expression Recognition: A Survey

Contact Info

Product

Resources

About