Speech emotion recognition approaches: A systematic review

Hashem, Ahlam; Arif, Muhammad; Alghamdi, Manal

doi:10.1016/j.specom.2023.102974

Cited by 16 publications

(3 citation statements)

References 128 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These advancements encompass various aspects, including emotional speech datasets, data enhancement and augmentation, feature extraction, and emotion classification. This comprehensive approach to SER underscores the field's movement towards more intricate and refined systems capable of interpreting a wider range of human emotions with greater accuracy [18].…”

Section: Comprehensive Analysis Of Current Speech Emotion Recognition...mentioning

confidence: 97%

Development of an advanced ai-based model for human psychoemotional state analysis

Ainakulov,

Koshekov,

Savostin

et al. 2023

EEJET

View full text Add to dashboard Cite

The research focuses on developing a novel method for the automatic recognition of human psychoemotional states (PES) using deep learning technology. This method is centered on analyzing speech signals to classify distinct emotional states. The primary challenge addressed by this research is to accurately perform multiclass classification of seven human psychoemotional states, namely joy, fear, anger, sadness, disgust, surprise, and a neutral state. Traditional methods have struggled to accurately distinguish these complex emotional nuances in speech. The study successfully developed a model capable of extracting informative features from audio recordings, specifically mel spectrograms and mel-frequency cepstral coefficients. These features were then used to train two deep convolutional neural networks, resulting in a classifier model. The uniqueness of this research lies in its use of a dual-feature approach and the employment of deep convolutional neural networks for classification. This approach has demonstrated high accuracy in emotion recognition, with an accuracy rate of 0.93 in the validation subset. The high accuracy and effectiveness of the model can be attributed to the comprehensive and synergistic use of mel spectrograms and mel-frequency cepstral coefficients, which provide a more nuanced analysis of emotional expressions in speech. The method presented in this research has broad applicability in various domains, including enhancing human-machine interface interactions, implementation in the aviation industry, healthcare, marketing, and other fields where understanding human emotions through speech is crucial

show abstract

Section: Comprehensive Analysis Of Current Speech Emotion Recognition...mentioning

confidence: 97%

Development of an advanced ai-based model for human psychoemotional state analysis

Ainakulov,

Koshekov,

Savostin

et al. 2023

EEJET

View full text Add to dashboard Cite

show abstract

“…It encompasses verbal and nonverbal modalities in daily communication, such as verbal communication, facial expression, tone of voice, hand movement, eye contact, and physiological response [8]. Technically speaking, emotion signals can be captured through audio, video, text, electroencephalogram (EEG), and other types of routine and clinical recordings [9].…”

Section: Introductionmentioning

confidence: 99%

Speech Emotion Recognition Using Dual-Stream Representation and Cross-Attention Fusion

Yu,

Meng,

Fan

et al. 2024

Electronics

View full text Add to dashboard Cite

Speech emotion recognition (SER) aims to recognize human emotions through in-depth analysis of audio signals. However, it remains challenging to encode emotional cues and to fuse the encoded cues effectively. In this study, dual-stream representation is developed, and both full training and fine-tuning of different deep networks are employed for encoding emotion patterns. Specifically, a cross-attention fusion (CAF) module is designed to integrate the dual-stream output for emotion recognition. Using different dual-stream encoders (fully training a text processing network and fine-tuning a pre-trained large language network), the CAF module is compared to other three fusion modules on three databases. The SER performance is quantified with weighted accuracy (WA), unweighted accuracy (UA), and F1-score (F1S). The experimental results suggest that the CAF outperforms the other three modules and leads to promising performance on the databases (EmoDB: WA, 97.20%; UA, 97.21%; F1S, 0.8804; IEMOCAP: WA, 69.65%; UA, 70.88%; F1S, 0.7084; RAVDESS: WA, 81.86%; UA, 82.75.21%; F1S, 0.8284). It is also found that fine-tuning a pre-trained large language network achieves superior representation than fully training a text processing network. In a future study, improved SER performance could be achieved through the development of a multi-stream representation of emotional cues and the incorporation of a multi-branch fusion mechanism for emotion recognition.

show abstract

“…Driver emotions play a vital role in ensuring road safety. Although in-car driver emotion detection has been extensively studied using facial expression analysis [1][2][3], speech emotion recognition [4][5][6], and physiological signal monitoring (e.g., movement, voice patterns, heart rate, brain activity, electrodermal activity) [7,8], personalised feedback mechanisms to help drivers regulate their emotions remain largely unexplored [9]. Timely, relevant, and actionable feedback is essential for drivers to adjust their behaviour and emotional state effectively.…”

Section: Introductionmentioning

confidence: 99%

Emotion-Aware In-Car Feedback: A Comparative Study

Mwaita,

Bhaumik,

Ahmed

et al. 2024

MTI

View full text Add to dashboard Cite

We investigate personalised feedback mechanisms to help drivers regulate their emotions, aiming to improve road safety. We systematically evaluate driver-preferred feedback modalities and their impact on emotional states. Using unobtrusive vision-based emotion detection and self-labeling, we captured the emotional states and feedback preferences of 21 participants in a simulated driving environment. Results show that in-car feedback systems effectively influence drivers’ emotional states, with participants reporting positive experiences and varying preferences based on their emotions. We also developed a machine learning classification system using facial marker data to demonstrate the feasibility of our approach for classifying emotional states. Our contributions include design guidelines for tailored feedback systems, a systematic analysis of user reactions across three feedback channels with variations, an emotion classification system, and a dataset with labeled face landmark annotations for future research.

show abstract

Speech emotion recognition approaches: A systematic review

Cited by 16 publications

References 128 publications

Development of an advanced ai-based model for human psychoemotional state analysis

Development of an advanced ai-based model for human psychoemotional state analysis

Speech Emotion Recognition Using Dual-Stream Representation and Cross-Attention Fusion

Emotion-Aware In-Car Feedback: A Comparative Study

Contact Info

Product

Resources

About