A Comparison of Machine Learning Algorithms and Feature Sets for Automatic Vocal Emotion Recognition in Speech

Abstract

Vocal emotion recognition (VER) in natural speech, often referred to as speech emotion recognition (SER), remains challenging for both humans and computers. Applied fields including clinical diagnosis and intervention, social interaction research or Human Computer Interaction (HCI) increasingly benefit from efficient VER algorithms. Several feature sets were used with machine-learning (ML) algorithms for discrete emotion classification. However, there is no consensus for which low-level-descriptors and classifiers are optimal. Therefore, we aimed to compare the performance of machine-learning algorithms with several different feature sets. Concretely, seven ML algorithms were compared on the Berlin Database of Emotional Speech: Multilayer Perceptron Neural Network (MLP), J48 Decision Tree (DT), Support Vector Machine with Sequential Minimal Optimization (SMO), Random Forest (RF), k-Nearest Neighbor (KNN), Simple Logistic Regression (LOG) and Multinomial Logistic Regression (MLR) with 10-fold cross validation using four openSMILE feature sets (i.e., IS-09, emobase, GeMAPS and eGeMAPS). Results indicated that SMO, MLP and LOG show better performance (reaching to 87.85%, 84.00% and 83.74% accuracies, respectively) compared to RF, DT, MLR and KNN (with minimum 73.46%, 53.08%, 70.65% and 58.69% accuracies, respectively). Overall, the emobase feature set performed best. We discuss the implications of these findings for applications in diagnosis, intervention or HCI.

Validation of scrambling methods for vocal affect bursts

“…Candidate sets of acoustic parameters have been identified for affective speech (e.g., Eyben et al, 2010 , 2016 ; Schuller et al, 2009 ) and nonverbal vocalizations (Sauter et al, 2010 ). However, testing responses to changing acoustic parameters and their combinations poses challenges for both human participants and machine learning algorithms (e.g., Doğdu et al, 2022 ) and is likely to be a focus of future research. Another approach to creating neutral versions of stimuli is to synthesize them.…”

Acoustic Features Distinguishing Emotions in Swedish Speech

“…Finally, it should be emphasized that this study focused Regarding methods, there is no consensus about which set of acoustic parameters are optimal for classifying emotions in speech. 33 GeMAPS 11 is developed to analyze affect and emotions in speech. It has been validated for analyses of sustained phonations as well as whole sentences.…”

A Roadmap for Technological Innovation in Multimodal Communication Research

“…For example, facial emotion expressions or speech were traditionally measured using electrodes or sensor coils (e.g., facial electromyography [172], real-time magnetic resonance imaging (MRI), or electromagnetic articulography [123], respectively). Although precise and wellestablished, such methods have the disadvantage of being bound to a lab than more recent contact-free approaches like video recordings for the analyses of facial expression, acoustic analysis for automatic speech recognition, and recognition of emotions in speech [40,170]. Video-based assessment of body pose is also increasingly used in clinical populations [111], however, developing efficient machine learning algorithms for semantic analysis of body pose and gestures remains a challenge.…”