Adversarial Auto-Encoders for Speech Based Emotion Recognition

Abstract

Recently, generative adversarial networks and adversarial autoencoders have gained a lot of attention in machine learning community due to their exceptional performance in tasks such as digit classification and face recognition. They map the autoencoder's bottleneck layer output (termed as code vectors) to different noise Probability Distribution Functions (PDFs), that can be further regularized to cluster based on class information. In addition, they also allow a generation of synthetic samples by sampling the code vectors from the mapped PDFs. Inspired by these properties, we investigate the application of adversarial auto-encoders to the domain of emotion recognition. Specifically, we conduct experiments on the following two aspects: (i) their ability to encode high dimensional feature vector representations for emotional utterances into a compressed space (with a minimal loss of emotion class discriminability in the compressed space), and (ii) their ability to regenerate synthetic samples in the original feature space, to be later used for purposes such as training emotion recognition classifiers. We demonstrate promise of adversarial auto-encoders with regards to these aspects on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) corpus and present our analysis.

Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG)

“…This allows for the intermediate representation to discover a more noise robust representation that can work well across domains. Further work has examined different variations of autoencoders for speech emotion recognition, including Variational Autoencoders (VAE) [28], [29], Adversarial Autoencoders (AAE) [29], [30], [31], and Adversarial Variational Bayes (AVB) [29].…”

“…The novelty of this paper includes: (1) the ADDoG model, which allows for better generalized representation convergence by "meeting in the middle"; (2) the MADDoG method, which extends ADDoG to allow for many dataset differences to be CycleGAN [16], [17], [18] DAA [4], [27] VAE [28], [29] AAE [29], [30], [31] AVB [29] ADDA [19] DANN [20], [21] ADDoG MADDoG Fig. 1.…”

Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives [Review Article]

“…For speech emotion recognition, researchers in [97] implemented an adversarial autoencoder model. In this work, highdimensional feature vectors of real data are encoded into 2-D dimensional representations, and a discriminator is learnt to distinguish real 2-D vectors from generated 2-D vectors.…”

“…The experiments indicate that the 2-D representations of real data can yield suitable margins between different emotion categories. Additionally, when adding the generated data to the original data for training, performance can be marginally increased [97].…”

Data Augmentation for Low Resource Sentiment Analysis Using Generative Adversarial Networks

“…I pursue a data augmentation approach to aid sentiment analysis in this paper. Data augmentation through GANs has been used in other tasks such as enhancing emotion recognition through speech [20,21], human pose estimation [22] and medical image synthesis [23]. In my work, I use a variant of cGAN and apply various heuristics to achieve their convergence.…”