Spotting Micro-Expressions on Long Videos Sequences

Abstract

This paper presents two methods for the first Micro-Expression Spotting Challenge 2019 by evaluating local temporal pattern (LTP) and local binary pattern (LBP) on two most recent databases, i.e. SAMM and CAS(ME) 2 . First we propose LTP-ML method as the baseline results for the challenge and then we compare the results with the LBP-χ 2distance method. The LTP patterns are extracted by applying PCA in a temporal window on several facial local regions. The micro-expression sequences are then spotted by a local classification of LTP and a global fusion. The LBP-χ 2 -distance method is to compare the feature difference by calculating χ 2 distance of LBP in a time window, the facial movements are then detected with a threshold. The performance is evaluated by Leave-One-Subject-Out cross validation. The overlap frames are used to determine the True Positives and the metric F1-score is used to compare the spotting performance of the databases. The F1-score of LTP-ML result for SAMM and CAS(ME) 2 are 0.0316 and 0.0179, respectively. The results show our proposed LTP-ML method outperformed LBP-χ 2 -distance method in terms of F1-score on both databases.

Micro-expression detection in long videos using optical flow and recurrent neural networks

“…Additionally, the number of false negatives (FN) is obtained by taking the total number of micro-expressions in the dataset, subtracted by the number of true positives. Our definition of a TP differs from the proposed definition of a TP in Li et al [10], in which the interval needs to be in proportion to the duration of the micro-expression. The definition of a TP described in Li et al would not work well for our method, because it currently focuses on spotting intervals of size |W |, which can be passed to annotators, rather than spotting the exact onset and offset of a micro-expression.…”

“…The proposed method performs quite well, albeit with a slightly different definition of a true positive. On all computed metrics it outperforms the baseline presented by Li et al [10]. To accurately compare to other state-of-the-art methods, we would need to use the exact same definitions.…”

“…Example of a short video with labeled onset, apex and offset frame, and an example of a long video that encompasses the short video. The sequence is extracted from the SAMM dataset, which was recorded at 200 fps [9] At the time of writing, only a limited number of published works have explored spotting in long video sequences [10], [11], [12], [13]. However, there have been other works that explored micro-expression spotting, but since they did not have suitable long video data available to them yet, it is difficult to tell how those methods would perform on the long video sequences provided in CAS(ME) 2 and SAMM.…”

MEGC 2019 – The Second Facial Micro-Expressions Grand Challenge

“…The lone submission for the spotting challenge was by Li et al [11], which proposed the use of local temporal patterns [12] (LTP-ML). The long video is divided into short clips by a sliding window.…”

Sliding Window Based Micro-expression Spotting: A Benchmark

“…Generic face preprocessing includes four steps: (1) Face detection and In existing studies, several methods utilized various face alignment steps that make the differences in the final face size and affect the performance of the later spotting and recognition steps. For example, several methods utilized the Discriminate response map fitting (DRMF) [27,31], while another studies select Active Shape Model [32,21] to extract the landmark points.…”