FACS-Based Graph Features for Real-Time Micro-Expression Recognition

Abstract

Several studies on micro-expression recognition have contributed mainly to accuracy improvement. However, the computational complexity receives lesser attention comparatively and therefore increases the cost of micro-expression recognition for real-time application. In addition, majority of the existing approaches required at least two frames (i.e., onset and apex frames) to compute features of every sample. This paper puts forward new facial graph features based on 68-point landmarks using Facial Action Coding System (FACS). The proposed feature extraction technique (FACS-based graph features) utilizes facial landmark points to compute graph for different Action Units (AUs), where the measured distance and gradient of every segment within an AU graph is presented as feature. Moreover, the proposed technique processes ME recognition based on single input frame sample. Results indicate that the proposed FACS-baed graph features achieve up to 87.33% of recognition accuracy with F1-score of 0.87 using leave one subject out cross-validation on SAMM datasets. Besides, the proposed technique computes features at the speed of 2 ms per sample on Xeon Processor E5-2650 machine.

Motion and Geometric Feature Analysis for Real-time Automatic Micro-expression Recognition Systems

“…However, the general problem with graph-based micro-expression recognition is the lack of large-scale in-the-wild datasets. To date, the recognition accuracy peaks at 87.33% over the SAMM dataset with LOSOCV as reported in Buhari et al 18…”

“…On the other hand, Table 4 lists the benchmark geometric-based methods [15][16][17][18][19] with the full-face graph and full-face graph + A-EMM, which are denoted as experiment I and experiment II, respectively. From these results, Buhari et al, 18 reported the highest accuracies of 76.67%, 75.04%, 81.41%, and 87.33% over the SMIC, CASMEII, CAS (ME) 2 , and SAMM datasets, respectively. In Buhari et al, 18 the full-face graph utilized 68 landmarks from the raw images (denoted as ℝ).…”

“…The BiWOOF is considered in this study as the results of its performance improvement over the textural feature extraction methods such as local binary patterns on three orthogonal planes (LBP-TOP) as reported in Liong et al 20 However, the computational cost poses challenges for real-time recognition as it require apex-frame spotting. The second feature descriptor considered is a full-face graph by Buhari et al 18 This geometric-based feature computation method requires only facial landmarks to compute features. The fullface graph is considered in this study because the computational time is significantly low in comparison with motionbased.…”

“…The indexes (i.e., p,q) of every landmark with subsequent landmark points are determined and stored as segments in ℑ for feature computations. After the graph is generated, features are computed by calculating the Euclidean distance and gradient of every segment, an idea presented in Buhari et al 18 The total number of features computed using this technique is N Â ðN À 1Þ ! K, which translates to 4,556 features at N ¼ 68.…”

“…[11][12][13][14] Towards facial landmark graph features, Lei et al, 15 presented a method that only employed 28 brow and lip landmarks, which contributed significantly to micro-expressions. While, other studies [16][17][18][19] presented graph-based methods using AU to define landmarks of interest. The reported recognition accuracies from these methods proved that micro-expression features can be extracted using facial graph approaches.…”