Facial expression recognition with trade-offs between data augmentation and deep learning features

Umer, Saiyed; Rout, Ranjeet Kumar; Pero, Chiara; Nappi, Michele

doi:10.1007/s12652-020-02845-8

Cited by 67 publications

(33 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most of the implementation cases of FERS, the facial region is analyzed as a texture where numerous techniques such as statistical and structural-based methods have been employed to extract discriminant features [14]. Apart from these techniques, recently, deep learning-based approaches with convolution neural networks [15] have been employed to extract more discriminant and distinctive features to ensure that a better performance can be obtained. However, most of these methods are database-dependent and these databases have been captured spontaneously under controlled environments [16] with tightly controlled illumination, age, and pose variation conditions.…”

Section: Fear Anger Disgust Happy Neutral Sad Surprisementioning

confidence: 99%

“…Earlier CNN models were used to solve character recognition tasks [24], but nowadays, CNN is widely used in various object recognition problems. Here, the most important ingredient for the success of CNN is the availability of large quantities of training data, i.e., the use of image augmentation techniques [15]. Additionally, the CNN achieves high performance by learning powerful high-level features by combining global appearances to local geometric features rather than conventional handcrafted features.…”

Section: Related Workmentioning

confidence: 99%

“…In machine learning, image augmentation techniques artificially increase the amount of training data by applying transformation methods to the existing data [63]. The classical augmentation techniques that were employed are bilateral filtering, unsharp filtering, horizontal flip, vertical flip, Gaussian blur, additive Gaussian noise, image scale, image cropping, translation, image rotation, shear mapping, image zooming, image filling, and contrast normalization methods from [15] for the purpose of image augmentation. The whole training images were flipped horizontally by applying simple image data augmentation techniques.…”

Section: Factors Affecting the Performance Of The Proposed Fersmentioning

confidence: 99%

See 2 more Smart Citations

A Unified Framework of Deep Learning-Based Facial Expression Recognition System for Diversified Applications

et al. 2021

Self Cite

View full text Add to dashboard Cite

This work proposes a facial expression recognition system for a diversified field of applications. The purpose of the proposed system is to predict the type of expressions in a human face region. The implementation of the proposed method is fragmented into three components. In the first component, from the given input image, a tree-structured part model has been applied that predicts some landmark points on the input image to detect facial regions. The detected face region was normalized to its fixed size and then down-sampled to its varying sizes such that the advantages, due to the effect of multi-resolution images, can be introduced. Then, some convolutional neural network (CNN) architectures were proposed in the second component to analyze the texture patterns in the facial regions. To enhance the proposed CNN model’s performance, some advanced techniques, such data augmentation, progressive image resizing, transfer-learning, and fine-tuning of the parameters, were employed in the third component to extract more distinctive and discriminant features for the proposed facial expression recognition system. The performance of the proposed system, due to different CNN models, is fused to achieve better performance than the existing state-of-the-art methods and for this reason, extensive experimentation has been carried out using the Karolinska-directed emotional faces (KDEF), GENKI-4k, Cohn-Kanade (CK+), and Static Facial Expressions in the Wild (SFEW) benchmark databases. The performance has been compared with some existing methods concerning these databases, which shows that the proposed facial expression recognition system outperforms other competing methods.

show abstract

Section: Fear Anger Disgust Happy Neutral Sad Surprisementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Factors Affecting the Performance Of The Proposed Fersmentioning

confidence: 99%

See 1 more Smart Citation

A Unified Framework of Deep Learning-Based Facial Expression Recognition System for Diversified Applications

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [34], the authors merged VGG with ResNet 50 to produce an output slightly better than both. Data augmentation has also been shown to improve the detection rates [42]- [44]. For example, [44] improved the accuracy by using rotations, translations, and other transformations on the original images to augment the size of the datasets.…”

Section: Hybrid Featuresmentioning

confidence: 99%

“…Neural networks are well known to require a large amount of data to fine tune. Data augmentation methods that increase the size of the dataset can lead to an improved accuracy, as seen in [43], [44], and [46]. To see the effect of data augmentation on the detection accuracy, we implemented 5 types of augmentation for every image in each of the datasets.…”

Section: B Data Augmentationmentioning

confidence: 99%

Mobile-Optimized Facial Expression Recognition Techniques

2021

View full text Add to dashboard Cite

This paper presents two novel facial expression recognition techniques: the real-time ensemble for facial expression recognition (REFER) and the facial expression recognition network (FERNet). Both approaches can detect facial expressions from various poses, distances, angles, and resolutions, and both techniques exhibit high computational efficiency and portability. REFER outperforms the existing approaches in terms of cross-dataset accuracy, making it an ideal network to use on fresh data. FERNet is a compact convolutional neural network that uses both geometric and texture features to achieve up to 98% accuracy on the MUG dataset. Both approaches can process 14 frames per second (FPS) from a live video capture on a battery-powered Raspberry Pi 4.INDEX TERMS Facial expression recognition, machine learning, multithreaded, active shape model, poseinvariant.

show abstract

Fused deep learning based Facial Expression Recognition of students in online learning mode

Sumalakshmi

Vasuki²

2022

Concurrency and Computation

View full text Add to dashboard Cite

In this research work, Facial Expression Recognition (FER) is used in the analysis of facial expressions during the online learning sessions in the prevailing pandemic situation. An integrated geometric and appearance feature extraction is presented for the FER of the students participating in the online classes. The integrated features provided a low-dimensional significant feature area for better facial data representation. Feasible Weighted Squirrel Search Optimization (FW-SSO) algorithm is applied for selecting the optimal features due to its efficient exploration of the search space and enhancement of the dynamic search. The output of the FW-SSO algorithm is used for tuning the autoencoder. Autoencoder is used for combining the G&A features, for feature optimization process. Classification is done by using Long Short-Term Memory (LSTM) network with Attention Mechanism (ALSTM), as it is highly efficient in capturing the long-term dependency of the facial landmarks in the image/video sequences.The proposed fused deep learning method focuses on the fusion of the G&A features for high discrimination. Experimental analysis using FER-2013 and LIRIS datasets demonstrated that the proposed method achieved maximum accuracy of 85.96% than the existing architectures and maximum accuracy of 88.24% than the VGGNet-CNN architecture.

show abstract

Facial expression recognition with trade-offs between data augmentation and deep learning features

Cited by 67 publications

References 50 publications

A Unified Framework of Deep Learning-Based Facial Expression Recognition System for Diversified Applications

A Unified Framework of Deep Learning-Based Facial Expression Recognition System for Diversified Applications

Mobile-Optimized Facial Expression Recognition Techniques

Fused deep learning based Facial Expression Recognition of students in online learning mode

Contact Info

Product

Resources

About