Recognizing spontaneous micro-expression from eye region

Duan, Xiaodong; Dai, Qigen; Wang, Xinhan; Wang, Yuangang; Hua, Zhichao

doi:10.1016/j.neucom.2016.03.090

Cited by 31 publications

(14 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…When compared with previous studies, as shown in Table 6, Duan also stated where the eye area was the most influential in each expression [28]. Wen-Jiang Yan's research, which uses FACS observations or based on AU movements, also states that the eye area also affects the expression of disgust and shock.…”

Section: Discussionmentioning

confidence: 88%

See 1 more Smart Citation

A Novel Approach on Motion Estimation for Micro-Expression Recognition Using Phase Only Correlation with All Block Search (POC-ABS)

Rosiani¹,

Choirina²,

Muna³

et al. 2020

IJIES

View full text Add to dashboard Cite

Micro-expression is an expression when a person tries to held or hidden, but the leak of this emotion still occurs in one or two areas of the face or maybe a short expression that across in the whole-face. Not more than 500ms, micro-expressions can be difficult to recognize and detect where the leakage area is located. This study presents a new method to recognize and detect the subtle motion on the facial components area using Phase Only Correlation algorithm with All Block Search (POC-ABS) to estimate the motion of all block areas. This block matching method is proposed by comparing each block in the two frames to determine whether there is movement or not. If the two blocks are identical, then the motion vector value is not displayed, whereas if the blocks are non-identical, the motion vector value of the POC is displayed. The motion vector, which is as a motion feature, estimates whether or not there are movements in the same block. In order to further confirm the reliability of the proposed method, two different classifiers were used for the micro-expression recognition of the CASME II dataset. The highest performance results are for SVM at 94.3 percent and for KNN at 95.6 percent. Finally, this algorithm detects leaks of motion based on the ratio of the motion vectors. The left and right eyebrows are dominant when expressing disgust, sadness, and surprise. Meanwhile, the movements of the right eye and left eye were the most dominant when the happiness expression.

show abstract

Section: Discussionmentioning

confidence: 88%

“…The observations are particularly prioritized on the eyes area and additionally the mouth area. Region-based research better results than global-based research (full-face observation) [28]. The motion features obtained from all areas of the facial components are used for the micro-expression classification.…”

Section: Introductionmentioning

confidence: 99%

A Novel Approach on Motion Estimation for Micro-Expression Recognition Using Phase Only Correlation with All Block Search (POC-ABS)

Rosiani¹,

Choirina²,

Muna³

et al. 2020

IJIES

View full text Add to dashboard Cite

show abstract

“…However, to date, there are several parts which could be used to enhance the performance of micro-expression recognition. Firstly, MEs are not distributed uniformly across the whole face [22]- [25], and they occur in a combination of several facial regions. For example, the muscular movements of the cheek lift and mouth upturn show a happy emotion, while the raised eyebrows and slightly parted lips indicate that someone feels surprised; Secondly, when using different feature descriptors to extract the block features, the dimension of the feature vectors will increase exponentially with the increase of parameters.…”

Section: Although Ekman and His Team Developed Micro Expressionmentioning

confidence: 99%

A New Framework Combining Local-Region Division and Feature Selection for Micro-Expressions Recognition

Zhang

Jiang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Micro-expressions are deliberate or unconscious movements of people's psychological activities, reflecting the transient facial true expressions. Previous works focus on the whole face for micro-expressions recognition. These methods can extract a number of feature vectors which are relevant or irrelevant to the micro-expressions recognition. Besides, the high-dimension feature vectors can result in longer computational time and increased computational complexity. In order to address these problems, we propose a new framework which combines the local-region division and the feature selection. Based on the proposed framework, the original images can retain more efficient regions and filter out the invalid components of feature vectors. Specifically, with the joint efforts of the facial deformation identification model and facial action coding system, the global region is divided into seven local regions with their corresponding actions units. The ReliefF algorithm is used to select effective components of feature vectors and reduce the dimension. To evaluate the proposed framework, we conduct experiments on both the Chinese Academy of Sciences Micro-expression II Database and Spontaneous Micro-expression Database with Leave-One-Subject-Out Cross Validation method. The results show that the performance in local combined regions outperforms its counterpart in the global region, and the recognition accuracy is further improved with the combination of feature selection.

show abstract

“…For addressing the problem of MER, many approaches including conventional and deep methods have been developed to model the fleeting subtle changes of spontaneous microexpressions towards the individual-database task. The conventional methods [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18] usually extract handcrafted features, e.g., local binary patterns on three orthogonal planes (LBP-TOP) [5], [8], second-order Gaussian jet on LBP-TOP [6], LBP six intersection points (LBP-SIP) [7], local spatiotemporal directional features (LSDF) [10], spatiotemporal LBP (STLBP) [9], spatiotemporal completed local quantization patterns (STCLQP) [12], discriminative spatiotemporal LBP (DSLBP) [16], directional mean optical-flow (MDMO) [14], bi-weighted oriented optical flow (Bi-WOOF) [17] and fuzzy histogram of optical flow orientation (FHOFO) [18], and then construct a classifier, e.g., support vector machine (SVM) [5], [11], [9], [12], [17] and random forest (RF) [5], [8], [13], [15], specially for MER. Although these handcrafted features continue to improve the representation ability for MER, it is still difficult to manually design good representations for capturing quick subtle changes of micro-expressions.…”

Section: Introductionmentioning

confidence: 99%

Revealing the Invisible With Model and Data Shrinking for Composite-Database Micro-Expression Recognition

Xia

Peng

Khor

et al. 2020

IEEE Trans. on Image Process.

117

View full text Add to dashboard Cite

Composite-database micro-expression recognition is attracting increasing attention as it is more practical to realworld applications. Though the composite database provides more sample diversity for learning good representation models, the important subtle dynamics are prone to disappearing in the domain shift such that the models greatly degrade their performance, especially for deep models. In this paper, we analyze the influence of learning complexity, including the input complexity and model complexity, and discover that the lower-resolution input data and shallower-architecture model are helpful to ease the degradation of deep models in composite-database task. Based on this, we propose a recurrent convolutional network (RCN) to explore the shallower-architecture and lower-resolution input data, shrinking model and input complexities simultaneously. Furthermore, we develop three parameter-free modules (i.e., wide expansion, shortcut connection and attention unit) to integrate with RCN without increasing any learnable parameters. These three modules can enhance the representation ability in various perspectives while preserving not-very-deep architecture for lower-resolution data. Besides, three modules can further be combined by an automatic strategy (a neural architecture search strategy) and the searched architecture becomes more robust. Extensive experiments on MEGC2019 dataset (composited of existing SMIC, CASME II and SAMM datasets) have verified the influence of learning complexity and shown that RCNs with three modules and the searched combination outperform the state-ofthe-art approaches.

show abstract

Recognizing spontaneous micro-expression from eye region

Cited by 31 publications

References 33 publications

A Novel Approach on Motion Estimation for Micro-Expression Recognition Using Phase Only Correlation with All Block Search (POC-ABS)

A Novel Approach on Motion Estimation for Micro-Expression Recognition Using Phase Only Correlation with All Block Search (POC-ABS)

A New Framework Combining Local-Region Division and Feature Selection for Micro-Expressions Recognition

Revealing the Invisible With Model and Data Shrinking for Composite-Database Micro-Expression Recognition

Contact Info

Product

Resources

About