Autonomous Emotion Learning in Speech: A View of Zero-Shot Speech Emotion Recognition

Abstract

Conventionally, speech emotion recognition is achieved using passive learning approaches. Differing from such approaches, we herein propose and develop a dynamic method of autonomous emotion learning based on zero-shot learning. The proposed methodology employs emotional dimensions as the attributes in the zero-shot learning paradigm, resulting in two phases of learning, namely attribute learning and label learning. Attribute learning connects the paralinguistic features and attributes utilising speech with known emotional labels, while label learning aims at defining unseen emotions through the attributes. The experimental results achieved on the CINEMO corpus indicate that zero-shot learning is a useful technique for autonomous speech-based emotion learning, achieving accuracies considerably better than chance level and an attribute-based gold-standard setup. Furthermore, different emotion recognition tasks, emotional attributes, and employed approaches strongly influence system performance.

Exploring Zero-Shot Emotion Recognition in Speech Using Semantic-Embedding Prototypes

“…Zero-Shot Speech Emotion Recognition: A basic framework for zero-shot learning in SER, containing two phases of attribute learning and label learning, is presented in [24]. The attribute-learning phase constructs the relationship between paralinguistic features and emotional descriptors or attributes, through the procedure of regression on seen-emotional samples using Support Vector Regression (SVR) or Deep Neural Networks (DNNs).…”

“…In order to deal with recognising unseen-emotional speech samples, we propose an autonomous learning strategy based on zero-shot emotion recognition [24]. Zero-Shot Learning (ZSL) has demonstrated high utility in image processing [25]- [28] and affective computing [29]- [31].…”

“…Zero-Shot Learning (ZSL) has demonstrated high utility in image processing [25]- [28] and affective computing [29]- [31]. We have also demonstrated a zero-shot framework for SER [24]. However, this framework required a high number of fully labelled emotional descriptors or attributes (i.e., emotional dimensions) which construct the label-learning models for seen-emotional samples.…”

SEMOUR: A Scripted Emotional Speech Repository for Urdu

“…It has been shown that the CNN module combined with long short-term memory (LSTM) neural networks works better than standalone CNN or LSTM based models for SER tasks in cross-corpus setting [37]. Other deep learning-based models like zero-shot learning, which learns using only a few labels [51] and Generative Adversarial Networks (GANs) to generate synthetic samples for robust learning have also been studied [10].…”

Identifying surgical-mask speech using deep neural networks on low-level aggregation

“…Computational paralinguistics make it possible to extract latent knowledge in audio signals (i.e., spoken signals) from human beings or animals [1][2][3]. Typical paralinguistics-related topics include emotion and personality recognition [4][5][6], autism diagnosis [7], native-speaker identification [8], or eating classification [9]. As an emerging topic in paralinguistics, Mask-Speech Identification (MSI) attempts to automatically distinguish whether a spoken utterance is pronounced by its speaker with or without a surgical mask [4].…”