Dynamic interactive multiview memory network for emotion recognition in conversation

Wen, Jintao; Jiang, Dong; Tu, Geng; Liu, Cheng; Wang, Zhaoxia

doi:10.1016/j.inffus.2022.10.009

Cited by 47 publications

(18 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The text, image, and common sentiment vectors that have embedded emotion [28] words (referred to as F T , F I , and F A ) are fed into a unique multi-head attention fusion module. This module aids in the integration of textual information into the model.…”

Section: Multi-head Attention Fusion Modulementioning

confidence: 99%

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Fu,

Liu,

Wang

et al. 2024

Electronics

Self Cite

View full text Add to dashboard Cite

Sarcasm poses a significant challenge for detection due to its unique linguistic phenomenon where the intended meaning is often opposite of the literal expression. Current sarcasm detection technology primarily utilizes multi-modal processing, but the connotative semantic information provided by the modality itself is limited. It is a challenge to mine the semantic information contained in the combination of sarcasm samples and external commonsense knowledge. Furthermore, as the essence of sarcasm detection lies in measuring emotional inconsistency, the rich semantic information may introduce excessive noise to inconsistency measurement. To mitigate these limitations, we propose a hierarchical framework in this paper. Specifically, to enrich the semantic information of each modality, our approach uses sentiment dictionaries to obtain the sentiment vectors by evaluating the words extracted from various modalities, and then combines them with each modality. Furthermore, in order to mine the joint semantic information implied in the modalities and improve measurement of emotional inconsistency, the emotional information representation obtained by fusing each modality’s data is concatenated with the sentiment vector. Then, cross-modal fusion is performed through cross-attention, and, finally, the sarcasm is recognized by fusing low-level information in the cross-modal fusion layer. Our model is evaluated on a public multi-modal sarcasm detection dataset based on Twitter, and the results demonstrate its superiority.

show abstract

Section: Multi-head Attention Fusion Modulementioning

confidence: 99%

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Fu,

Liu,

Wang

et al. 2024

Electronics

Self Cite

View full text Add to dashboard Cite

show abstract

“…A plethora of studies have been significantly impacted by the use of sentiment analysis approaches to analyse the content of social [ 27 , 28 ]. This is accomplished by inferring implicit emotions from users' textual content on online social sites [ 29 , 30 ].…”

Section: Introductionmentioning

confidence: 99%

Emotion detection of social data: APIs comparative study

Abu-Salih

Alhabashneh

Zhu

et al. 2023

Heliyon

View full text Add to dashboard Cite

“…Emotions are complex and expressed through various modalities, including facial expressions, gestures, and oral expressions ( Liu P. et al, 2022 ). Scholars agree that emotions consist of multiple components and are thus complex to study ( Gao and Cui, 2022 ; Liu et al, 2023 ; Wen et al, 2023 ). Several studies claim that a single modality is insufficient for providing a comprehensive understanding of emotions due to their complexity ( Goncalves et al, 2017 ; Liu W. et al, 2022 ).…”

Section: Introductionmentioning

confidence: 99%

Multimodality in language education: implications of a multimodal affective perspective in foreign language teaching

Guo

2023

Front. Psychol.

View full text Add to dashboard Cite

Foreign language learners often encounter various emotional challenges within academic environments, which can hinder their progress in developing literacy skills. Effective language instruction should encompass teaching approaches that acknowledge the emotional requirements of students. To address this need, we propose a multimodal affective methodology designed to evaluate emotions in foreign language education scenarios. This methodology also holds the potential to elucidate the pedagogic contributions of various emotional variables to academic outcomes. Our study focuses on German as a foreign language (GFL) learning and utilizes it as an example to investigate ways to improve writing proficiency. The study explores the effects of integrating multimodal corrective feedback (MCF) into academic writing exercises. We delve into suitable modalities for analyzing emotions in academic writing practices. Furthermore, we investigate how the choice of corrective feedback mode intricately influences the nature of feedback itself and subsequently influences students’ emotional responses. Through a comprehensive exploration of the interplay between distinct modes of delivering feedback and their impacts on learners’ engagement, this investigation strives to decode the intricate dynamics of emotions that underlie language acquisition. With these insights, the study discusses how teachers can enhance their teaching strategies by combining changes in learners’ emotional states and providing emotional support.

show abstract

Dynamic interactive multiview memory network for emotion recognition in conversation

Cited by 47 publications

References 50 publications

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Emotion detection of social data: APIs comparative study

Multimodality in language education: implications of a multimodal affective perspective in foreign language teaching

Contact Info

Product

Resources

About