Unsupervised Cross-Lingual Speech Emotion Recognition Using Domain Adversarial Neural Network

Cai, Xiong; Wu, Zhiyong; Zhong, Kuo; Su, Bin; Dai, Dongyang; Meng, Helen

doi:10.1109/iscslp49672.2021.9362058

Cited by 10 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They evaluate the proposed architecture against cross-corpus settings and achieve improved results compared to AE-based models and Danns. In [165], the authors evaluate a DANN against cross-lingual SER. They use GRL with a language classifier, which helps the model to learn language-independent emotional representations.…”

Section: Deep Domain Adaptive Representation Learningmentioning

confidence: 99%

Survey of Deep Representation Learning for Speech Emotion Recognition

Latif

Rana

Khalifa

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Traditionally, speech emotion recognition (SER) research has relied on manually handcrafted acoustic features using feature engineering. However, the design of handcrafted features for complex SER tasks requires significant manual effort, which impedes generalisability and slows the pace of innovation. This has motivated the adoption of representation learning techniques that can automatically learn an intermediate representation of the input signal without any manual feature engineering. Representation learning has led to improved SER performance and enabled rapid innovation. Its effectiveness has further increased with advances in deep learning (DL), which has facilitated deep representation learning where hierarchical representations are automatically learned in a data-driven manner. This paper presents the first comprehensive survey on the important topic of deep representation learning for SER. We highlight various techniques, related challenges and identify important future areas of research. Our survey bridges the gap in the literature since existing surveys either focus on SER with hand-engineered features or representation learning in the general setting without focusing on SER.

show abstract

Section: Deep Domain Adaptive Representation Learningmentioning

confidence: 99%

Survey of Deep Representation Learning for Speech Emotion Recognition

Latif

Rana

Khalifa

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…It is designed to address challenges related to training deep neural networks by normalizing activations within smaller groups instead of the entire batch. GN does not impact cross-domain transfer learning and can save memory [15]. The mathematical formulation of Group Normalization is given by:…”

Section: Self-normalization Network Blockmentioning

confidence: 99%

Feature Selection and Fusion in Cantonese Speech Emotion Analysis

2023

AJCIS

View full text Add to dashboard Cite

This work addresses the scarcity of Cantonese speech emotion datasets by introducing a dedicated dataset and employing innovative methodologies. A tailored feature set, specifically designed for Cantonese, captures intricate emotional expressions. Enhanced efficiency in Cantonese speech emotion recognition is showcased through the utilization of a self-normalization network-based model. With an impressive accuracy of 92.3% on the Cantonese dataset, the model demonstrates robust generalization capabilities across diverse Chinese and English datasets. The obtained results underscore the potential applications of this research in various domains, including Cantonese language education, psychological counseling, and voice assistants. Understanding of Cantonese emotional expressions is advanced, contributing to the preservation of linguistic and cultural heritage.Despite the notable achievements, limitations in dataset coverage and emotion variety are acknowledged. Future endeavors will prioritize expanding the dataset's breadth and incorporating a wider range of emotional expressions. Additionally, the exploration of more comprehensive Cantonese emotion recognition will involve the investigation of multimodal approaches, where audio, visual, and textual cues are combined. These efforts are aimed at addressing current limitations and pushing the field toward a more nuanced understanding of Cantonese emotional communication.

show abstract

“…Previous studies have identified several popular feature extraction techniques for SER, including low-level descriptors (LLDs), the mel spectrogram, the wav2vec representation [7], and feature selection based on genetic algorithms [8][9][10][11]. LLDs have been used in studies [12][13][14], while the mel spectrogram has been used in studies such as [15][16][17][18][19]. Wav2vec, on the other hand, has been used in studies [20][21][22][23][24][25].…”

Section: Introductionmentioning

confidence: 99%

“…The third observation is that no studies have been conducted on cross-lingual SER using S3R features to date. The features commonly used in the community, such as the mel spectrogram [16], usually contain some unhelpful information for SER, such as language information. In contrast, language information may even degrade performance.…”

Section: Introductionmentioning

confidence: 99%

Single- and Cross-Lingual Speech Emotion Recognition Based on WavLM Domain Emotion Embedding

Yang,

Liu,

Huang

et al. 2024

Electronics

View full text Add to dashboard Cite

Unlike previous approaches in speech emotion recognition (SER), which typically extract emotion embeddings from a trained classifier consisting of fully connected layers and training data without considering contextual information, this research introduces a novel approach. It integrates contextual information into the feature extraction process. The proposed approach is based on the WavLM representation and incorporates a contextual transform, along with fully connected layers, training data, and corresponding label information, to extract single-lingual WavLM domain emotion embeddings (SL-WDEEs) and cross-lingual WavLM domain emotion embeddings (CL-WDEEs) for single-lingual and cross-lingual SER, respectively. To extract CL-WDEEs, multi-task learning is employed to remove language information, marking it as the first work to extract emotion embeddings for cross-lingual SER. Experimental results on the IEMOCAP database demonstrate that the proposed SL-WDEE outperforms some commonly used features and known systems, while results on the ESD database indicate that the proposed CL-WDEE effectively recognizes cross-lingual emotions and outperforms many commonly used features.

show abstract

Unsupervised Cross-Lingual Speech Emotion Recognition Using Domain Adversarial Neural Network

Cited by 10 publications

References 26 publications

Survey of Deep Representation Learning for Speech Emotion Recognition

Survey of Deep Representation Learning for Speech Emotion Recognition

Feature Selection and Fusion in Cantonese Speech Emotion Analysis

Single- and Cross-Lingual Speech Emotion Recognition Based on WavLM Domain Emotion Embedding

Contact Info

Product

Resources

About