Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture

Zhang, Hepeng; Huang, Bin; Tian, Guohui

doi:10.1016/j.patrec.2019.12.013

Cited by 70 publications

(31 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We incorporate LSTM in the network architecture. LSTM has been widely used in the task of facial expression and action recognition [29][30][31][32]. Zhang et al [31] combined the time and texture information of image sequences by Total paralysis Ⅲ…”

Section: Related Workmentioning

confidence: 99%

Region Based Parallel Hierarchy Convolutional Neural Network for Automatic Facial Nerve Paralysis Evaluation

Liu

Xia

et al. 2020

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Region Based Parallel Hierarchy Convolutional Neural Network for Automatic Facial Nerve Paralysis Evaluation

Liu

Xia

et al. 2020

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

“…In [35], the concept of attention is introduced to the first layer of the convolutional neural network to perform convolution calculations of ROIs. In [36][37], a multichannel convolutional neural network was used for feature fusion. In [36], a dual-channel weighted hybrid deep convolutional neural network (WMDCNN) based on static images and a dual-channel weighted hybrid deep long short-term memory network based on image sequences (WMCNN-LSTM) were proposed.…”

Section: B Convolutional Neural Network Methodsmentioning

confidence: 99%

A Facial Expression Recognition Method Based on a Multibranch Cross-Connection Convolutional Neural Network

2021

View full text Add to dashboard Cite

This paper proposes a new method based on a multiple branch cross-connected convolutional neural network (MBCC-CNN) for facial expression recognition. Compared with traditional machine learning methods, the proposed method can extract image features more effectively. In addition, in contrast to single-structure convolutional neural networks, the MBCC-CNN model is constructed based on the residual connection, Network in Network, and tree structure approaches together. It also adds a shortcut cross connection for the summation of the convolution output layer, which makes the data flow between networks more smooth, improves the feature extraction ability of each receptive field. The proposed method can fuse the features of each branches more effectively, which solves the problem of insufficient feature extraction of each branches and increases the recognition performance. The experimental results based on the Fer2013, CK+, FER+ and RAF data sets show that the recognition rates of the proposed MBCC-CNN method are 71.52%, 98.48%, 88.10% and 87.34%, respectively. Compared with some most recently work, the proposed method can provide better facial expression recognition performance and has good robustness. INDEX TERMS Facial expression recognition; Convolutional neural network; Residual connection; Network in Network; Robustness.

show abstract

“…Although these approaches are effective methods in extracting spatial information, they fail to capture morphological and contextual variations in the expression process. Recent methods aim to solve this problem by using massive datasets to obtain more efficient features of FER [9][10][11][12][13][14][15]. Some researchers use multimodal fusion to recognize emotions, such as voices, expressions, and actions [16].…”

Section: Related Workmentioning

confidence: 99%

Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

Zhang

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

show abstract

Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture

Cited by 70 publications

References 18 publications

Region Based Parallel Hierarchy Convolutional Neural Network for Automatic Facial Nerve Paralysis Evaluation

Region Based Parallel Hierarchy Convolutional Neural Network for Automatic Facial Nerve Paralysis Evaluation

A Facial Expression Recognition Method Based on a Multibranch Cross-Connection Convolutional Neural Network

Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

Contact Info

Product

Resources

About