Spatio-temporal fusion for Macro- and Micro-expression Spotting in Long Video Sequences

Zhang, Liwei; Li, Jingting; Wang, Sujing; Duan, Xiaojie; Yan, Wen-Jing; Xie, Haiyong; Huang, Shucheng

doi:10.1109/fg47880.2020.00037

Cited by 50 publications

(14 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The state of the art techniques arose from the recent microexpression spotting competition, the Micro-Expression Grand Challenge 2020 [17]. We found that our model is comparable to the existing techniques, being beaten only by the optical flow based technique proposed by Zhang et al [39]. The F 1 scores reported by Pan are averaged scores for both microexpression and macroexpression spotting and thus should not be compared directly with the other methods [23].…”

Section: Microexpressionsmentioning

confidence: 80%

Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results

Speth¹,

Vance²,

Czajka³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present the Deception Detection and Physiological Monitoring (DDPM) dataset and initial baseline results on this dataset. Our application context is an interview scenario in which the interviewee attempts to deceive the interviewer on selected responses. The interviewee is recorded in RGB, near-infrared, and long-wave infrared, along with cardiac pulse, blood oxygenation, and audio. After collection, data were annotated for interviewer/interviewee, curated, ground-truthed, and organized into train / test parts for a set of canonical deception detection experiments. Baseline experiments found random accuracy for micro-expressions as an indicator of deception, but that saccades can give a statistically significant response. We also estimated subject heart rates from face videos (remotely) with a mean absolute error as low as 3.16 bpm. The database contains almost 13 hours of recordings of 70 subjects, and over 8 million visible-light, near-infrared, and thermal video frames, along with appropriate meta, audio and pulse oximeter data. To our knowledge, this is the only collection offering recordings of five modalities in an interview scenario that can be used in both deception detection and remote photoplethysmography research.

show abstract

Section: Microexpressionsmentioning

confidence: 80%

Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results

Speth¹,

Vance²,

Czajka³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The co-occurrence of macro-and micro-expressions are common in real life. An automatic spotting system for micro-and macro-expressions was designed by [53]. A new spotting benchmark has been proposed recently [118].…”

Section: Micro-expression Spottingmentioning

confidence: 99%

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

Xie¹,

Lo²,

Shuai³

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Facial micro-expressions indicate brief and subtle facial movements that appear during emotional communication. In comparison to macro-expressions, micro-expressions are more challenging to be analyzed due to the short span of time and the fine-grained changes. In recent years, micro-expression recognition (MER) has drawn much attention because it can benefit a wide range of applications, e.g. police interrogation, clinical diagnosis, depression analysis, and business negotiation. In this survey, we offer a fresh overview to discuss new research directions and challenges these days for MER tasks. For example, we review MER approaches from three novel aspects: macro-to-micro adaptation, recognition based on key apex frames, and recognition based on facial action units. Moreover, to mitigate the problem of limited and biased ME data, synthetic data generation is surveyed for the diversity enrichment of micro-expression data. Since micro-expression spotting can boost micro-expression analysis, the state-of-the-art spotting works are also introduced in this paper. At last, we discuss the challenges in MER research and provide potential solutions as well as possible directions for further investigation.

show abstract

“…Prior to the learning phase, a series of pre-processing steps are introduced to ensure consistency of the data before model learning. Motivated by the work of [19], we take the landmark position of the nose region with five pixels margin to eliminate the global head motion for each frame. Then, we omit the left and right eye regions since optical flow features are highly sensitive to eye blinking [9].…”

Section: Pre-processingmentioning

confidence: 99%

Shallow Optical Flow Three-Stream CNN for Macro- and Micro-Expression Spotting from Long Videos

Liong¹,

See²,

Wong³

2021

Preprint

View full text Add to dashboard Cite

Facial expressions vary from the visible to the subtle. In recent years, the analysis of micro-expressions-a natural occurrence resulting from the suppression of one's true emotions, has drawn the attention of researchers with a broad range of potential applications. However, spotting microexpressions in long videos becomes increasingly challenging when intertwined with normal or macro-expressions. In this paper, we propose a shallow optical flow three-stream CNN (SOFTNet) model to predict a score that captures the likelihood of a frame being in an expression interval. By fashioning the spotting task as a regression problem, we introduce pseudo-labeling to facilitate the learning process. We demonstrate the efficacy and efficiency of the proposed approach on the recent MEGC 2020 benchmark, where state-of-the-art performance is achieved on CAS(ME) 2 with equally promising results on SAMM Long Videos.

show abstract

Spatio-temporal fusion for Macro- and Micro-expression Spotting in Long Video Sequences

Cited by 50 publications

References 20 publications

Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results

Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results

An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge

Shallow Optical Flow Three-Stream CNN for Macro- and Micro-Expression Spotting from Long Videos

Contact Info

Product

Resources

About