Joint Local and Global Information Learning With Single Apex Frame Detection for Micro-Expression Recognition

Li, Yante; Huang, Xiaohua; Zhao, Guoying

doi:10.1109/tip.2020.3035042

Cited by 105 publications

(68 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Through multiple feature transformations, the original data features can be transformed into effective feature representations. Finally input into the classification model to get the recognition result (Li et al, 2021). As the basic model of deep learning, Convolutional Neural Networks (CNN), combined with network models such as Recurrent Neural Network (RNN) and Long Short Term Memory Network (LSTM), have been widely used in emotion recognition.…”

Section: Related Researchmentioning

confidence: 99%

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Liu

Wang

et al. 2021

Front. Neurorobot.

View full text Add to dashboard Cite

The redundant information, noise data generated in the process of single-modal feature extraction, and traditional learning algorithms are difficult to obtain ideal recognition performance. A multi-modal fusion emotion recognition method for speech expressions based on deep learning is proposed. Firstly, the corresponding feature extraction methods are set up for different single modalities. Among them, the voice uses the convolutional neural network-long and short term memory (CNN-LSTM) network, and the facial expression in the video uses the Inception-Res Net-v2 network to extract the feature data. Then, long and short term memory (LSTM) is used to capture the correlation between different modalities and within the modalities. After the feature selection process of the chi-square test, the single modalities are spliced to obtain a unified fusion feature. Finally, the fusion data features output by LSTM are used as the input of the classifier LIBSVM to realize the final emotion recognition. The experimental results show that the recognition accuracy of the proposed method on the MOSI and MELD datasets are 87.56 and 90.06%, respectively, which are better than other comparison methods. It has laid a certain theoretical foundation for the application of multimodal fusion in emotion recognition.

show abstract

Section: Related Researchmentioning

confidence: 99%

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Liu

Wang

et al. 2021

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…In the face of what relatives do to make them uncomfortable, people will not have more patience with relatives as in traditional cognition, but are more likely to be irritated by what close people do not conform to their wishes. [ 29 ] Through interviews, we learn that a gentleman who has been careful at work has never been in conflict with his colleagues or boss in decades of work. In the eyes of his colleagues and boss, he is a man who has never been in conflict with or anxious.…”

Section: Exploring the Reasons Why People Are More Likely To Spread Anxiety At Homementioning

confidence: 99%

Investigation and Analysis of Anxiety Dissemination in Various Situations and Research on Methods of Eliminating Anxiety in Population

Cao

Zhang

2021

Preprint

View full text Add to dashboard Cite

It is very common that people are affected by different levels of anxiety in different environments. However, there is no effective research or corresponding measures for this phenomenon. This study through the questionnaire survey combined with interviews and experiments, through the investigation and experiment to find out the reasons for the spread of anxiety in the group, and through further experiments found that people can produce positive emotions and eliminate anxiety when watching portraits with positive emotions， and finally find out the method to solve the problem of anxiety in private and public occasions.

show abstract

“…The efficiency of ME apex frame detector is further improved using HOOF (Histogram Oriented Optical Flow) features [22]. [23] used the 3D FFT (Fast Fourier Transform) method for ME detection. The Methods achieved good results but they merely considered comparative small changes with respect to neighboring frames.…”

Section: Literature Reviewmentioning

confidence: 99%

“…But still, these method does not perform significantly and still lot of improvement can be done. Recently many deep-learning algorithms [4], [23], [27]- [29] are also used for ME classification. Most of the method used transfer learning, CNN (Convolutional Neural Networks) architecture trained on special features like optical flow [28] and FFT features [23].…”

Section: Literature Reviewmentioning

confidence: 99%

“…Recently many deep-learning algorithms [4], [23], [27]- [29] are also used for ME classification. Most of the method used transfer learning, CNN (Convolutional Neural Networks) architecture trained on special features like optical flow [28] and FFT features [23]. While some method used pre-trained CNN models for classification [4], [29].…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

EFFICIENT METHODS FOR FACIAL MICRO-EXPRESSIONS DETECTION & CLASSIFICATION

Yadav¹,

Priyanka²,

Kacker³

2021

INDJCSE

View full text Add to dashboard Cite

Micro-expressions (ME) are a form of facial expression that lasts just a few milliseconds. Due to its use in revealing implicit human intentions, particularly in high-stakes situations, ME detection and classification is a burgeoning field of research. In recent studies, it has been found that the apex frame contains maximum facial movement conveying expression information. But they last for very few milliseconds, hence detecting the apex frame and classifying the ME is a very challenging task. This paper addresses this problem by proposing a training-free method for ME detection and an ME classifier using hybrid features. For ME detection, video frames are preprocessed to remove head movement and align faces. Then, from five local facial regions, Local Binary Patterns (LBP) are extracted. These regions are selected to reduce the effects of head eye blinks and changing eye gaze. Then the apex frame is detected using the Frobenius norm on extracted LBP features. For ME classification, hybrid features: Frame Difference (FD), Optical Flow Magnitude (OF-Mag), and Optical Flow Histogram of Oriented Gradients are extracted from regions around facial landmarks and are used to classify micro-expressions. These regions are selected to reduce the effects of head movement, eye blinks, and changing eye gaze. The proposed methods are tested on publicly available benchmark datasets CASME II. On the CASME II, the ME detection method achieved 0.1137 Normalized Mean Absolute Error (NMAE), 0.0720 Normalized Standard Error (NSE), and the ME classifier achieved 67.07 percent and 0.69, accuracy and a f1-score respectively. The results reveal that the proposed methods excelled the existing methods.

show abstract

Joint Local and Global Information Learning With Single Apex Frame Detection for Micro-Expression Recognition

Cited by 105 publications

References 43 publications

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Investigation and Analysis of Anxiety Dissemination in Various Situations and Research on Methods of Eliminating Anxiety in Population

EFFICIENT METHODS FOR FACIAL MICRO-EXPRESSIONS DETECTION & CLASSIFICATION

Contact Info

Product

Resources

About