Classifying Emotions and Engagement in Online Learning Based on a Single Facial Expression Recognition Neural Network

Savchenko, Andrey V.; Savchenko, L. V.; Makarov, Ilya

doi:10.1109/taffc.2022.3188390

Cited by 163 publications

(57 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The current state of the art for emotion recognition of Affectnet dataset is [14], proposed face detection, tracking, and clustering techniques which are applied to extract the sequences of faces from each frame. Next, a single efficient neural network is used to extract emotional features in each frame.…”

Section: Related Workmentioning

confidence: 99%

Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformers

Vats¹,

Chadha²

2023

Preprint

View full text Add to dashboard Cite

We present a facial emotion recognition framework, built upon Swin vision Transformers jointly with squeeze and excitation block (SE). A transformer model based on an attention mechanism has been presented recently to address vision tasks. Our method uses a vision transformer with a Squeeze excitation block (SE) and sharpness-aware minimizer (SAM). We have used a hybrid dataset, to train our model and the AffectNet dataset to evaluate the result of our model.

show abstract

Section: Related Workmentioning

confidence: 99%

Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformers

Vats¹,

Chadha²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…We follow [51] for 8 emotion classification, where emotion label is given for each video frame. We further apply sliding window smoothing algorithm [52] on the temporal domain to smooth the distribution of emotion along time.…”

Section: A4 Selected Algorithmsmentioning

confidence: 99%

CelebV-HQ: A Large-Scale Video Facial Attributes Dataset

Zhu

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

“…This module allows us to combine the results of several statistical functions (such as minimum, maximum, mean value, and standard deviation) calculated component by component when using the features of all the video frames. As a result, we obtain a descriptor, which contains information about all the input frames [23]. We apply the L 2 -norm to the descriptors and use the result to train the SVM with linear kernel that predicts one of the classes of emotions.…”

Section: Proposed Approachmentioning

confidence: 99%

Neural Network Model for Video-Based Analysis of Student’s Emotions in E-Learning

Savchenko

Makarov²

2022

Opt. Mem. Neural Networks

View full text Add to dashboard Cite

In this paper, we consider a problem of an automatic analysis of the emotional state of students during online classes based on video surveillance data. This problem is actual in the field of e-learning. We propose a novel neural network model for recognition of students’ emotions based on video images of their faces and use it to construct an algorithm for classifying the individual and group emotions of students by video clips. At the first step, it performs detection of the faces and extracts their features followed by grouping the face of each student. To increase the accuracy, we propose to match students’ names selected with the aid of the algorithms of the text recognition. At the second step, specially learned efficient neural networks perform the extraction of emotional features of each selected person, their aggregation with the aid of statistical functions, and the subsequent classification. At the final step, it is possible to visualize fragments of the video lesson with the most pronounced emotions of the student. Our experiments with some datasets from EmotiW (Emotion Recognition in the Wild) show that the accuracy of the developed algorithms is comparable with their known analogous. However, when classifying emotions, the computational performance of these algorithms is higher.

show abstract

Classifying Emotions and Engagement in Online Learning Based on a Single Facial Expression Recognition Neural Network

Cited by 163 publications

References 52 publications

Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformers

Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformers

CelebV-HQ: A Large-Scale Video Facial Attributes Dataset

Neural Network Model for Video-Based Analysis of Student’s Emotions in E-Learning

Contact Info

Product

Resources

About