Geometrical Features and Active Appearance Model Applied to Facial Expression Recognition

Abstract

One of the most effective ways of expressing emotion is through facial expressions. This work proposes and discusses a geometrical descriptor based on the calculation of distances from coordinates of facial fiducial points, which are used as features for training support vector machines (SVM) to classify emotions. Three data sets are studied and six basic emotions are considered in our experiments. In comparison to other approaches available in the literature, the results obtained with our geometrical descriptor demonstrated to be very competitive, achieving high classification F-score rates. Additionally, we evaluate whether the combination of our geometrical descriptor with an appearance feature, the Gabor filter, allows emotions to be even more distinguishable for the classifier. The result is positive for two out of three data sets. Finally, to simulate in-the-wild scenarios, an active appearance model (AAM) is trained to position the fiducial points on the correct facial locations, instead of using the ones provided by the data sets. As the fitting error is considered acceptable, the former experiments are also conducted with the new data generated by the AAM. The results show a small drop on the F-score values when compared to the data originally provided by the data sets,but are still satisfactory.

Deep Stacked Autoencoder-Based Automatic Emotion Recognition Using an Efficient Hybrid Local Texture Descriptor

“…The geometric method uses the absolute and relative geometrical relationship of facial components for feature extraction. Two well-known model-based facial feature point detection methods are an Active Shape Model (ASM) (Cootes, Taylor, Cooper, & Graham,1995;Shbib, & Zhou, 2015) and an Active Appearance Model (AAM) (Edwards, Cootes, & Taylor, 1998;Maximiano da Silva, &Pedrini, 2016). The main drawback of the statistical model is the model fitting.…”

Modelling and Analysis of Facial Expressions Using Optical Flow Derived Divergence and Curl Templates

“…Static-or frame-based methods categorize the emotion from still images embodying the momentary appearance of the FE, generally in its peak form. Silva et al [17] presented a compact and effective description of face depicting an FE based on horizontal and vertical distance between distinct fiducial points whose locations were known a priori. In one of the experiments, they integrated this geometric feature vector with Gabor filters to extract appearance features and demonstrated that complementing the two features enabled a better discrimination between different emotions.…”

Human expression recognition using facial shape based Fourier descriptors fusion

“…First, geometric features based on the edges of specific fiducial points from different facial parts and second, appearance based features to represent the muscular texture of the face using change of wrinkles and creases. Study of the FER methods, applied on geometric based features are Active Appearance Model (AAM) [6], elastic bunch graph matching (EBGM) [7], and straight-line distances [8] of fiducial points. On the other hand, appearance based feature extraction techniques are Local Fisher Discriminant Analysis (LFDA) [9], local binary patterns (LBP) [10] and recently Convolutional Neural Networks (CNN) [11].…”