Multi-task Micro-expression Recognition Combining Deep and Handcrafted Features

Hu, Chunlong; Jiang, Dengbiao; Zou, Haitao; Zuo, Xin; Shu, Yucheng

doi:10.1109/icpr.2018.8545555

Cited by 16 publications

(14 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recognition rate (%) MMEW SAMM FDM [84] 34.6 34.1 ResNet10 [117] 36.6 39.3 Handcrafted features + deep features [118] 36.6 47.1 LBP-TOP [40] 38.9 37.0 Selective deep features [93] 39.0 42.9 ELRCN [97] 41.5 46.2 DCP-TOP [42] 42.5 36.8 ESCSTF [119] 42.7 46.9 LHWP-TOP [42] 43.2 41.7 LBP-MOP [45] 43.9 35.3 LBP-SIP [44] 43.9 37.4 DiSTLBP-RIP [53] 44.0 46.2 RHWP-TOP [42] 45.9 38.1 STLBP-IP [52] 46.6 42.9 ApexME [120] 48.8 50.0 Transfer Learning [121] 52.4 55.9 Multi-task mid-level feature learning [62] 54.2 55.0 KGSL [61] 56.9 48.6 Sparse MDMO [86] 60.0 52.9 MDMO [85] 65.7 50.0 DTSCNN [94] 65.9 69.2 TLCNN [96] 69.4 73.5…”

Section: Methodsmentioning

confidence: 99%

“…Setting 2. For handcrafted features + deep features [118], we employ two scales (10, 20 pixels) and three orientations (0, 60 and 120 degrees), resulting in 6 different Gabor filters, and set the sizes of radii on the three orthogonal planes of LBP-TOP to (1, 1, 2). The deep CNN contains 5 convolutional layers and 3 fully connected layers.…”

Section: Comparisons Of State-of-the-art Methodsmentioning

confidence: 99%

“…The output sizes of the two fully connected layers are 128 and 6 respectively.Setting 2. For handcrafted features + deep features[118], we employ two scales(10, 20 pixels…”

mentioning

confidence: 99%

See 2 more Smart Citations

Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms

Ben

Ren²,

Zhang³

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

139

View full text Add to dashboard Cite

Unlike the conventional facial expressions, micro-expressions are involuntary and transient facial expressions capable of revealing the genuine emotions that people attempt to hide. Therefore, they can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, however, their detection and recognition is difficult and relies heavily on expert experiences. Due to its intrinsic particularity and complexity, video-based micro-expression analysis is attractive but challenging, and has recently become an active area of research. Although there have been numerous developments in this area, thus far there has been no comprehensive survey that provides researchers with a systematic overview of these developments with a unified evaluation. Accordingly, in this survey paper, we first highlight the key differences between macro-and micro-expressions, then use these differences to guide our research survey of video-based micro-expression analysis in a cascaded structure, encompassing the neuropsychological basis, datasets, features, spotting algorithms, recognition algorithms, applications and evaluation of state-of-the-art approaches. For each aspect, the basic techniques, advanced developments and major challenges are addressed and discussed. Furthermore, after considering the limitations of existing micro-expression datasets, we present and release a new dataset -called micro-and-macro expression warehouse (MMEW)containing more video samples and more labeled emotion types. We then perform a unified comparison of representative methods on CAS(ME) 2 for spotting, and on MMEW and SAMM for recognition, respectively. Finally, some potential future research directions are explored and outlined.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Comparisons Of State-of-the-art Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms

Ben

Ren²,

Zhang³

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

139

View full text Add to dashboard Cite

show abstract

“…LBP-TOP variants. Aside from the use of original LBP-TOP operator, several variants were proposed to meet different needs for MER [52], [78], [79], [80], [47], [46], [81]. Huang et al [52] combined the idea of integral projection and texture descriptor like LBP-TOP to bone texture characterization and face recognition.…”

Section: Handcrafted Featurementioning

confidence: 99%

“…Guo et al [79] extended the LBP-TOP operator into two novel binary descriptors: Angular Difference LBP-TOP (ADLBPTOP) and Radial Difference LBP-TOP (RDLBPTOP), respectively. Hu et al [80] combined LBP-TOP and learning based features to form a feature fusion for multi-task learning. Hu et al [47] also took advantage of Gabor filter and proposed Centralized Gabor Binary Pattern from Three Orthogonal Panels (CGBP-TOP).…”

Section: Handcrafted Featurementioning

confidence: 99%

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

Xie¹,

Lo²,

Shuai³

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Facial micro-expressions indicate brief and subtle facial movements that appear during emotional communication. In comparison to macro-expressions, micro-expressions are more challenging to be analyzed due to the short span of time and the fine-grained changes. In recent years, micro-expression recognition (MER) has drawn much attention because it can benefit a wide range of applications, e.g. police interrogation, clinical diagnosis, depression analysis, and business negotiation. In this survey, we offer a fresh overview to discuss new research directions and challenges these days for MER tasks. For example, we review MER approaches from three novel aspects: macro-to-micro adaptation, recognition based on key apex frames, and recognition based on facial action units. Moreover, to mitigate the problem of limited and biased ME data, synthetic data generation is surveyed for the diversity enrichment of micro-expression data. Since micro-expression spotting can boost micro-expression analysis, the state-of-the-art spotting works are also introduced in this paper. At last, we discuss the challenges in MER research and provide potential solutions as well as possible directions for further investigation.

show abstract

An Improved DCNN Based Facial Micro-expression Recognition System

Garg

Verma

2021

Transactions on Computer Systems and Networks

View full text Add to dashboard Cite

Multi-task Micro-expression Recognition Combining Deep and Handcrafted Features

Cited by 16 publications

References 28 publications

Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms

Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

An Improved DCNN Based Facial Micro-expression Recognition System

Contact Info

Product

Resources

About