Fine Tuning Vision Transformer Model for Facial Emotion Recognition: Performance Analysis for Human-Machine Teaming

Roka, Sanjeev; Rawat, Danda B.

doi:10.1109/iri58017.2023.00030

Cited by 1 publication

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Apart from CNN based models, many research have been done using attention or transformer based approaches for Facial Emotion Recognition Task to enhance human computer interaction. 17 One of the approaches by, 18 trains a plain vision transformer model on a hybrid dataset (merging FER-2013, 6 CK+48 19 and AffectNet 20 forming AVFER). The author prepares a balanced dataset combining three different datasets along with different augmentation techniques and uses pre-trained ViT models to classify the facial emotions.…”

Section: Related Workmentioning

confidence: 99%

Improved face transformer model for facial emotion recognition in human-machine collaboration

Roka,

Rawat

2024

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications VI

View full text Add to dashboard Cite

With the spread of powerful AI models, human-machine interaction has improved rapidly. In this situation, recognizing facial expressions of human emotion is essential for successful human-machine collaboration/ teaming. While Convolutional Neural Network (CNN) models were widely used for facial emotion classification, Transformer-based models, known for excelling in NLP tasks, have demonstrated superior performance in areas like image classification, semantic segmentation, and object detection. This study investigates the effectiveness of using a transformer based model, the Face-transformer model a powerful tool for facial identification, that will be fine-tuned for the Facial Emotion Recognition (FER) challenge. We aim to modify the face transformer architecture to recognize emotional states from facial photos using the extensive facial emotion recognition datasets, opening the door for more natural and responsive machine interactions. Our initial findings suggest that the Face-Transformer model holds promise for bridging the gap between machine interpretability and human emotions, potentially paving the way for more natural and responsive human-computer interactions.

show abstract

Section: Related Workmentioning

confidence: 99%