Action unit detection using sparse appearance descriptors in space-time video volumes

Abstract

Recently developed appearance descriptors offer the opportunity for efficient and robust facial expression recognition. In this paper we investigate the merits of the family of local binary pattern descriptors for FACS Action-Unit (AU) detection. We compare Local Binary Patterns (LBP) and Local Phase Quantisation (LPQ) for static AU analysis. To encode facial expression dynamics, we extend the purely spatial representation LPQ to a dynamic texture descriptor which we call Local Phase Quantisation from Three Orthogonal Planes (LPQ-TOP), and compare this with the Local Binary Patterns from Three Orthogonal Planes (LBP-TOP). The efficiency of these descriptors is evaluated by a fully automatic AU detection system and tested on posed and spontaneous expression data collected from the MMI and SEMAINE databases. Results show that the systems based on LPQ achieve higher accuracy rate than those using LBP, and that the systems that utilise dynamic appearance descriptors outperform those that use static appearance descriptors. Overall, our proposed LPQ-TOP method outperformed all other tested methods.

Facial Micro-Expression Detection in Hi-Speed Video Based on Facial Action Coding System (FACS)

“…The recognition rate of the 11 AUs for micro expressions analyzed by us are similar and sometimes better than the recognition for the same AUs for full expression using other state-of-the-art approaches [7], [29] and comparable to more recent works such [16], [17], [30].…”

“…Pantic group proposed the use of dynamic descriptors for analysis of facial texture changes in videos [16], [17]. They also compared Motion History Images (MHI) and FreeForm Deformations (FFDs) descriptors, GenleBoost being used as a classifier [16].…”

“…In addition, this work defines an approach for comparing results in expression phase detection. In [17] the family of local binary pattern descriptors for FACS AU detection are compared.…”

Unlock Screen Application Design Using Face Expression on Android Smartphone

“…In this study [5,8], there is a new approach to the analysis of the temporal dynamics explicitly produced by facial movements using a descriptor that can describe the dynamic changes in the face which is called Local Phase of Three Orthogonal Planes Quantisation (LPQ-TOP). Until now, most of the facial expression recognition system using only a static descriptor, where if there is a change in the movement of the face and other relevant information is ignored.…”

Binary Pattern Analysis for 3D Facial Action Unit Detection

“…Variants on LBPs have also been proposed to improve on the performance of the basic feature: Local Gradient Orientation Binary Patterns (LGOBPs) [10], Local Phase Quantisers (LPQ) [14], Local Gabor Binary Patterns (LGBPs) [30], and Histogram of Monogenic Binary Patterns (HMBPs) [29]. LBPs, LPQs and LGBPs have also been extended to the dynamic problem in the form of LBP-TOP [31], LPQ-TOP [7] and V-LGBPs [28]. The traditional LBP descriptor has also been applied to the depth map of a 3D facial mesh in 3DLBPs [6] and the Multi-resolution Extended Local Binary Pattern (MELBPs) [5].…”