Time efficient real time facial expression recognition with CNN and transfer learning

Podder, Tanusree; Bhattacharya, Diptendu; Majumdar, Angshuman

doi:10.1007/s12046-022-01943-x

Cited by 13 publications

(3 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Presented in [27], the sequential recurrent convolution network (SRCN) constitutes a two-part model, integrating a convolutional neural network (CNN) for feature extraction from input data and a sequence of long short-term memory (LSTM) models to capture temporal dynamics. In the study outlined in [28], the authors propose a method named LiveEmoNet, which synergistically combines a convolutional neural network (CNN) with minimal parameters and transfer learning (TL) techniques for facial expression recognition.…”

Section: Related Workmentioning

confidence: 99%

Augmentation-Enhanced Deep Learning for Face Detection and Emotion Recognition in Elderly Care Robots

Tran,

Phan,

Nguyen

et al. 2024

Preprint

View full text Add to dashboard Cite

Facial recognition and emotion detection are crucial for meaningful interactions between elderly individuals and healthcare robots. These technologies allow the robot to identify and understand facial expressions, adapting its responses accordingly. In our research, we propose an innovative approach that combines CNN-based face detection with LBC (Local Binary Convolution) for deep-learning emotion classification. We use a BlazeFace-based face detection model and a robust LBC-Xception emotion classification model. Additionally, we introduce a Multi-Augmentation data method, including seven basic data augmentation techniques and a combination of image super-resolution (ISR) and Conditional Generating Adversarial Network (cGAN) to enhance system robust-ness and accuracy. This combined approach not only improves accuracy through advanced feature extraction capabilities but also provides a lightweight network , suitable for hardware-constrained devices. To validate our method, we will compare face recognition and classification accuracy using the BlazeFace model against other popular classification models like MobileNetV2, VGG16, and Incep-tionV3. We’ll also compare datasets, including FER2013, FER2013Plus, and our augmented dataset (FER2013-MultiX). Finally, we develop an elderly care robot (ECR) and a robot control web application (RCWA) on the Robot Operating System 2 (ROS2) to evaluate performance under real conditions.

show abstract

Section: Related Workmentioning

confidence: 99%

Augmentation-Enhanced Deep Learning for Face Detection and Emotion Recognition in Elderly Care Robots

Tran,

Phan,

Nguyen

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…This technique has been used in recent studies for FSA. For example, in a study, the authors used facial landmarks to recognise emotions from YouTube videos to develop an effective feature selection algorithm to determine the optimal features for further improving the performance of multimodal sentiment analysis [74]. Another study utilised facial landmarks for emotion recognition in the context of social robotics [75].…”

Section: (C)mentioning

confidence: 99%

Emotions Matter: A Systematic Review and Meta-Analysis of the Detection and Classification of Students’ Emotions in STEM during Online Learning

Anwar,

Rehman,

Nasralla

et al. 2023

Education Sciences

View full text Add to dashboard Cite

In recent years, the rapid growth of online learning has highlighted the need for effective methods to monitor and improve student experiences. Emotions play a crucial role in shaping students’ engagement, motivation, and satisfaction in online learning environments, particularly in complex STEM subjects. In this context, sentiment analysis has emerged as a promising tool to detect and classify emotions expressed in textual and visual forms. This study offers an extensive literature review using the preferred reporting items for systematic reviews and meta-analyses (PRISMA) technique on the role of sentiment analysis in student satisfaction and online learning in STEM subjects. The review analyses the applicability, challenges, and limitations of text- and facial-based sentiment analysis techniques in educational settings by reviewing 57 peer-reviewed research articles out of 236 articles, published between 2015 and 2023, initially identified through a comprehensive search strategy. Through an extensive search and scrutiny process, these articles were selected based on their relevance and contribution to the topic. The review’s findings indicate that sentiment analysis holds significant potential for improving student experiences, encouraging personalised learning, and promoting satisfaction in the online learning environment. Educators and administrators can gain valuable insights into students’ emotions and perceptions by employing computational techniques to analyse and interpret emotions expressed in text and facial expressions. However, the review also identifies several challenges and limitations associated with sentiment analysis in educational settings. These challenges include the need for accurate emotion detection and interpretation, addressing cultural and linguistic variations, ensuring data privacy and ethics, and a reliance on high-quality data sources. Despite these challenges, the review highlights the immense potential of sentiment analysis in transforming online learning experiences in STEM subjects and recommends further research and development in this area.

show abstract

“…Artificial neurons have a wide range of applications, including image classification, recognition, and segmentation. Additionally, artificial neurons can perform simple convolutions [7]. Increasing the amount of data provided to the convolutional neural network can lead to the development of a more reliable and accurate deep learning model.…”

mentioning

confidence: 99%

The Implementation of Transfer Learning by Convolution Neural Network (CNN) for Recognizing Facial Emotions

Anbananthan Pillai Munanday,

Norazlianie Sazali,

Arjun Asogan

et al. 2023

ARASET

View full text Add to dashboard Cite

The primary objective of this study is to develop a real-time system that can predict the emotional states of an individual who commonly undergoes various experiences. The primary methodology suggested in this research for detecting facial expressions involves the integration of transfer learning techniques that incorporate convolutional neural networks (CNNs), along with a parameterization approach that minimizes the number of parameters. The FER-2013, JAFFE, and CK+ datasets were jointly used to train the CNN architecture for real-time detection, which broadened the range of emotional expressions that may be recognized. The proposed model has the capability to identify various emotions, including but not limited to happiness, fear, surprise, anger, contempt, sadness, and neutrality. Several methods were employed to assess the efficacy of the model's performance in this study. The experimental results indicate that the proposed approach surpasses previous studies in terms of both speed and accuracy.

show abstract

Time efficient real time facial expression recognition with CNN and transfer learning

Cited by 13 publications

References 45 publications

Augmentation-Enhanced Deep Learning for Face Detection and Emotion Recognition in Elderly Care Robots

Augmentation-Enhanced Deep Learning for Face Detection and Emotion Recognition in Elderly Care Robots

Emotions Matter: A Systematic Review and Meta-Analysis of the Detection and Classification of Students’ Emotions in STEM during Online Learning

The Implementation of Transfer Learning by Convolution Neural Network (CNN) for Recognizing Facial Emotions

Contact Info

Product

Resources

About