Speaker to Emotion: Domain Adaptation for Speech Emotion Recognition with Residual Adapters

Xi, Yuxuan; Li, Peng-Cheng; Yan, Shuicheng; Jiang, Yiheng; Dai, Li-Rong

doi:10.1109/apsipaasc47483.2019.9023339

Cited by 20 publications

(13 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that Xi et al [76] proposed a residual adapter model to recognize speech emotions on the CHEAVD2.0 dataset, and gives an accuracy of 43.96%, whereas our method presented an accuracy of 43.85%. Nevertheless, the used speaker adaption method in [76] is complicated due to its computation complexity. In addition, due to the used spontaneous emotional datasets, the reported performance on these three datasets is relatively still low.…”

Section: From the Results Ofmentioning

confidence: 74%

See 1 more Smart Citation

Speech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance

et al. 2020

View full text Add to dashboard Cite

Section: From the Results Ofmentioning

confidence: 74%

“…This shows the effectiveness of the used attention mechanism and data balance in our method. Besides, our method obtains a little lower performance than [76] on the CHEAVD2.0 dataset.…”

Section: From the Results Ofmentioning

confidence: 89%

Speech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance

et al. 2020

View full text Add to dashboard Cite

“…The studies reviewed in the previous section highlight the possibility of learning movement representations that can be transferred between tasks or datasets [1]. Studies such as [24], [25] further demonstrate advantage in applying representation learning to leverage related datasets for affect recognition tasks in particular.…”

Section: Leveraging Related Datasets For Affect Recognitionmentioning

confidence: 99%

Movement Representation Learning for Pain Level Classification

Olugbade,

Williams,

Gold

et al. 2024

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Self-supervised learning has shown value for uncovering informative movement features for human activity recognition. However, there has been minimal exploration of this approach for affect recognition where availability of large labelled datasets is particularly limited. In this paper, we propose a P-STEMR (Parallel Space-Time Encoding Movement Representation) architecture with the aim of addressing this gap and specifically leveraging the higher availability of human activity recognition datasets for pain-level classification. We evaluated and analyzed the architecture using three different datasets across four sets of experiments. We found statistically significant increase in average F1 score to 0.84 for pain level classification with two classes based on the architecture compared with the use of hand-crafted features. This suggests that it is capable of learning movement representations and transferring these from activity recognition based on data captured in lab settings to classification of pain levels with messier real-world data. We further found that the efficacy of transfer between datasets can be undermined by dissimilarities in population groups due to impairments that affect movement behaviour and in motion primitives (e.g. rotation versus flexion). Future work should investigate how the effect of these differences could be minimized so that data from healthy people can be more valuable for transfer learning.

show abstract

“…Researchers have explored domain adaptation techniques to improve the cross-corpus SER performance [4], [5], [6], [7]. Domain adaptation is a transfer learning approach where a trained model undergoes training aiming to optimise for a different distribution other than the previously trained distribution.…”

Section: Introductionmentioning

confidence: 99%

Domain Adapting Speech Emotion Recognition modals to real-world scenario with Deep Reinforcement Learning

Rajapakshe¹,

Rana²,

Khalifa³

2022

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning has been a popular training paradigm as deep learning has gained popularity in the field of machine learning. Domain adaptation allows us to transfer knowledge learnt by a model across domains after a phase of training. The inability to adapt an existing model to a real-world domain is one of the shortcomings of current domain adaptation algorithms. We present a deep reinforcement learning-based strategy for adapting a pre-trained model to a newer domain while interacting with the environment and collecting continual feedback. This method was used on the Speech Emotion Recognition task, which included both cross-corpus and cross-language domain adaption schema. Furthermore, it demonstrates that in a real-world environment, our approach outperforms the supervised learning strategy by 42% and 20% in cross-corpus and cross-language schema, respectively.

show abstract

Speaker to Emotion: Domain Adaptation for Speech Emotion Recognition with Residual Adapters

Cited by 20 publications

References 20 publications

Speech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance

Speech Emotion Recognition by Combining a Unified First-Order Attention Network With Data Balance

Movement Representation Learning for Pain Level Classification

Domain Adapting Speech Emotion Recognition modals to real-world scenario with Deep Reinforcement Learning

Contact Info

Product

Resources

About