A Multibias-Mitigated and Sentiment Knowledge Enriched Transformer for Debiasing in Multimodal Conversational Emotion Recognition

Wang, Jinglin; Ma, Fang; Zhang, Yazhou; Song, Dawei

doi:10.1007/978-3-031-17120-8_39

Cited by 3 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, Transformer based approaches aim to leverage the advance of Transformer architecture to model the contextual feature. For example, Wang et al (2022a) proposed to mitigate multi-bias knowledge from Transformer for emotion recognition in conversations. They proposed a series of approaches to mitigate five typical kinds of bias in textual utterances (i.e., gender, age, race, religion and LGBTQ+) and visual representations (i.e, gender and age).…”

Section: Multi-modal Emotion Recognitionmentioning

confidence: 99%

Multi-Modal Attentive Prompt Learning for Few-shot Emotion Recognition in Conversations

Liang,

Tu,

et al. 2024

jair

View full text Add to dashboard Cite

Emotion recognition in conversations (ERC) has emerged as an important research area in Natural Language Processing and Affective Computing, focusing on accurately identifying emotions within the conversational utterance. Conventional approaches typically rely on labeled training samples for fine-tuning pre-trained language models (PLMs) to enhance classification performance. However, the limited availability of labeled data in real-world scenarios poses a significant challenge, potentially resulting in diminished model performance. In response to this challenge, we present the Multi-modal Attentive Prompt (MAP) learning framework, tailored specifically for few-shot emotion recognition in conversations. The MAP framework consists of four integral modules: multi-modal feature extraction for the sequential embedding of text, visual, and acoustic inputs; a multi-modal prompt generation module that creates six manually-designed multi-modal prompts; an attention mechanism for prompt aggregation; and an emotion inference module for emotion prediction. To evaluate our proposed model’s efficacy, we conducted extensive experiments on two widely recognized benchmark datasets, MELD and IEMOCAP. Our results demonstrate that the MAP framework outperforms state-of-the-art ERC models, yielding notable improvements of 3.5% and 0.4% in micro F1 scores. These findings highlight the MAP learning framework’s ability to effectively address the challenge of limited labeled data in emotion recognition, offering a promising strategy for improving ERC model performance.

show abstract

Section: Multi-modal Emotion Recognitionmentioning

confidence: 99%