Multimodal Dialogue Understanding via Holistic Modeling and Sequence Labeling

Cai, Chenran; Zhao, Qin; Xu, Ruifeng; Qin, Bing

doi:10.1007/978-3-031-44699-3_36

Lecture Notes in Computer Science

2023

DOI: 10.1007/978-3-031-44699-3_36

|View full text |Cite

Multimodal Dialogue Understanding via Holistic Modeling and Sequence Labeling

Chenran Cai,

Qin Zhao,

Ruifeng Xu

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Fu,

Liu,

Wang

et al. 2024

Electronics

View full text Add to dashboard Cite

Sarcasm poses a significant challenge for detection due to its unique linguistic phenomenon where the intended meaning is often opposite of the literal expression. Current sarcasm detection technology primarily utilizes multi-modal processing, but the connotative semantic information provided by the modality itself is limited. It is a challenge to mine the semantic information contained in the combination of sarcasm samples and external commonsense knowledge. Furthermore, as the essence of sarcasm detection lies in measuring emotional inconsistency, the rich semantic information may introduce excessive noise to inconsistency measurement. To mitigate these limitations, we propose a hierarchical framework in this paper. Specifically, to enrich the semantic information of each modality, our approach uses sentiment dictionaries to obtain the sentiment vectors by evaluating the words extracted from various modalities, and then combines them with each modality. Furthermore, in order to mine the joint semantic information implied in the modalities and improve measurement of emotional inconsistency, the emotional information representation obtained by fusing each modality’s data is concatenated with the sentiment vector. Then, cross-modal fusion is performed through cross-attention, and, finally, the sarcasm is recognized by fusing low-level information in the cross-modal fusion layer. Our model is evaluated on a public multi-modal sarcasm detection dataset based on Twitter, and the results demonstrate its superiority.

show abstract

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Fu,

Liu,

Wang

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Multimodal Dialogue Understanding via Holistic Modeling and Sequence Labeling

Cited by 1 publication

References 23 publications

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

Contact Info

Product

Resources

About