Facial Expression Recognition Using Dynamic Local Ternary Patterns With Kernel Extreme Learning Machine Classifier

Abstract

Rapid growth in advanced human-computer interaction (HCI) based applications has led to the immense popularity of facial expression recognition (FER) research among computer vision and pattern recognition researchers. Lately, a robust texture descriptor named Dynamic Local Ternary Pattern (DLTP) developed for face liveness detection has proved to be very useful in preserving facial texture information. The findings motivated us to investigate DLTP in more detail and examine its usefulness in the FER task. To this end, a FER pipeline is developed, which uses a sequence of steps to detect possible facial expressions in a given input image. Given an input image, the pipeline first locates and registers faces in it. In the next step, using an image enhancement operator, the FER pipeline enhances the facial images. Afterward, from the enhanced images, facial features are extracted using the DLTP descriptor. Subsequently, the pipeline reduces dimensions of the high-dimensional DLTP features via Principal Component Analysis (PCA). Finally, using the multi-class Kernel Extreme Learning Machine (K-ELM) classifier, the proposed FER scheme classifies the features into facial expressions. Extensive experiments performed on four in-the-lab and one in-the-wild FER datasets confirmed the superiority of the method. Besides, the cross-dataset experiments performed on different combinations of the FER datasets revealed its robustness. Comparison results with several state-of-the-art FER methods demonstrate the usefulness of the proposed FER scheme. The pipeline with a recognition accuracy of 99.76%, 99.72%, 93.98%, 96.71%, and 78.75%, respectively, on the CK+, RaF, KDEF, JAFFE, and RAF-DB datasets, outperformed the previous state-of-the-art.INDEX TERMS Facial expression recognition, dynamic local ternary pattern, principal component analysis, kernel extreme learning machine, cross-dataset, cross-validation.

Symmetric Multi-Scale Residual Network Ensemble with Weighted Evidence Fusion Strategy for Facial Expression Recognition

“…The traditional feature engineering-based methods, composed of facial feature extraction and facial expression classification, mainly explore facial expression variations using geometric and appearance features [11], and classify facial expression categories based on support vector machine (SVM) [24] or principal component analysis (PCA) [25]. In general, geometric feature-based methods have the advantages of low dimension and being insensitive to illumination variations, but their ability for local detail description is weak.…”

Joint Transfer Extreme Learning Machine with Cross-Domain Mean Approximation and Output Weight Alignment

“…In equation (10), the frst two items are the loss of ELM, and the third item is the loss of CDMA in the output layer. α 1 is a tradeof parameter between two losses.…”

“…Tere is no need for ELM to tune input weight and bias, and what it only needs is to optimize the output weight by solving a least square problem. Terefore, it has been widely recognized for classifcation and regression in various felds including industry fault diagnosis [5,6], medical diagnosis [7], hyperspectral imagery classifcation [8,9], facial expression recognition [10], and brain-computer interface [11,12]. However, like the traditional machine learning model, ELM performs less satisfyingly when the training samples are insufcient.…”

Discriminative Extreme Learning Machine with Cross‐Domain Mean Approximation for Unsupervised Domain Adaptation

“…Classifiers based on machine learning play an important role in data and information processing systems. Extreme Learning Machine (ELM) [1] attracts attention due to its faster learning and higher accuracy compared with k Nearest Neighbor (kNN) [2], Back-Propagating (BP) [3], Naive Bayes (NB) [4], Support Vector Machine (SVM) [5], and Decision Tree (DT) [6], and has been widely promoted in many fields including image classification [7], traffic system [8], COVID-19 detection [9], fault diagnosis [10,11], hyperspectral remote sensing images [12,13], industrial sensors [14], facial expression recognition [15], and braincomputer interface (BCI) [16,17] etc.…”