Speaker-Invariant Adversarial Domain Adaptation for Emotion Recognition

Yin, Yufeng; Huang, Baiyu; Wu, Yizhen; Soleymani, Mohammad

doi:10.1145/3382507.3418813

Cited by 29 publications

(14 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The best result for the subject-invariant model was obtained with λ = 0.2, and the best result for the subject-invariant and domain adapted model was obtained with λ s = 0.15 and λ d = 0.1 (see Equation 1 and 3). We compared the performance of our method with a EEG CNN encoder [7], DANN [9] and SIDANN [8]. The results are summarized in Table 1.…”

Section: Resultsmentioning

confidence: 99%

“…The between- database results show that subject-invariant learning, in addition to domain adaptation methods, can partially address the lack of generalization in EEG-base emotion recognition. Our proposed method significantly outperforms SIDANN [8] (significance is tested via a t-test p − value < .05).…”

Section: Resultsmentioning

confidence: 99%

“…In this paper, inspired by the speaker invariant emotion recognition method proposed by Yin et al [8], we developed an unsupervised adversarial method to reduce the subjectspecific information of EEG signals and improve EEG-based emotion recognition performance. Unlike [8] that discriminates between all subjects, we introduce a discriminator between in-group (training subjects) and out-group (test subjects). This adversarial method is trained to learn a representation from EEG signals that minimizes emotion recognition loss while maximizing subject confusion loss, via a Gradient Reversal Layer (GRL).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Subject-Invariant Eeg Representation Learning For Emotion Recognition

Rayatdoost

Yin

Rudrauf

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

The discrepancies between the distributions of the train and test data, a.k.a., domain shift, result in lower generalization for emotion recognition methods. One of the main factors contributing to these discrepancies is human variability. Domain adaptation methods are developed to alleviate the problem of domain shift, however, these techniques while reducing between database variations fail to reduce between-subject variability. In this paper, we propose an adversarial deep domain adaptation approach for emotion recognition from electroencephalogram (EEG) signals. The method jointly learns a new representation that minimizes emotion recognition loss and maximizes subject confusion loss. We demonstrate that the proposed representation can improve emotion recognition performance within and across databases.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Subject-Invariant Eeg Representation Learning For Emotion Recognition

Rayatdoost

Yin

Rudrauf

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Second, the privacy filter (PF) obfuscates the user's personal information such as face, eye, age, gender, and emotion, using data manipulations such as 'deepfake' approaches [25,33,41] (Figure 1). We select these filters following previous works [1,45] that highlight the risks of video analytics (e.g., emotion recognition [44]) [45] and users concerns in ubiquitous environments regarding the types of collected data [1]. Moreover, these filters are easier for the participants to understand our prototype.…”

Section: System Overviewmentioning

confidence: 99%

PARA: Privacy Management and Control in Emerging IoT Ecosystems using Augmented Reality

Fernandez

Lee

Nurmi

et al. 2021

Proceedings of the 2021 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

The ubiquity of smart devices, combined with a lack of information about data garnered by them, make privacy a significant challenge for adopting smart devices. Ensuring users can safeguard their privacy without compromising the devices' functionality requires CCS CONCEPTS• Security and privacy → Usability in security and privacy; • Human-centered computing → Mixed / augmented reality; Graphical user interfaces.

show abstract

“…Although considerable research effort has been put into developing models capable of recognizing and predicting human emotions, much of the work focuses on lab-produced datasets [3] and robust speech emotion recognition on in-thewild speech with diverse confounding acoustic elements remains a challenge [32]. Previous work has used a variety of machine learning techniques such as multi-task learning [32] and domain generalization [31] that attempt to learn a more robust and generalizable representation of speech features, yet few studies have successfully accounted for variations in speech emotion corpora due to the complexity of emotion expressions and a variety of moderating variables such as gender, culture, etc.…”

Section: Introductionmentioning

confidence: 99%

Accounting for Variations in Speech Emotion Recognition with Nonparametric Hierarchical Neural Network

Lance¹,

Romana²,

Provost³

2021

Preprint

View full text Add to dashboard Cite

In recent years, deep-learning-based speech emotion recognition models have outperformed classical machine learning models. Previously, some neural network designs, such as Multitask Learning, have accounted for variations in emotional expressions due to demographic and contextual factors. However, existing models face a few constraints: 1) they rely on a clear definition of domains (e.g. gender, culture, noise condition, etc.) and the availability of domain labels. 2) they often attempt to learn domain-invariant features while emotion expressions can be domain-specific. In the present study, we propose the Nonparametric Hierarchical Neural Network (NHNN), a lightweight hierarchical neural network model based on Bayesian nonparametric clustering. In comparison to Multitask Learning approaches, the proposed model does not require domain/task labels. In our experiments, the NHNN models outperform the models with similar levels of complexity and state-of-the-art models in within-corpus and crosscorpus tests. Through clustering analysis, we show that the NHNN models are able to learn group-specific features and bridge the performance gap between groups.

show abstract

Speaker-Invariant Adversarial Domain Adaptation for Emotion Recognition

Cited by 29 publications

References 30 publications

Subject-Invariant Eeg Representation Learning For Emotion Recognition

Subject-Invariant Eeg Representation Learning For Emotion Recognition

PARA: Privacy Management and Control in Emerging IoT Ecosystems using Augmented Reality

Accounting for Variations in Speech Emotion Recognition with Nonparametric Hierarchical Neural Network

Contact Info

Product

Resources

About