Target-oriented multimodal sentiment classification by using topic model and gating mechanism

Song, Zhengxin; Xue, Yun; Gu, Donghong; Zhang, Haolan; Ding, Weiping

doi:10.1007/s13042-022-01757-7

Cited by 9 publications

(1 citation statement)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the contrary, an aspect itself in some degree can represent what it means even without its context. Because of this obvious difference, previous studies show their interest in differently modelling the two tasks to capture target related context (Xu et al, 2019;Yu et al, 2020;Song et al, 2023). For example, in Figure 1(a), "Chuck Bass", "MCM", and "Iran" do not have any exact meaning when they are disconnected from the specific context.…”

Section: Introductionmentioning

confidence: 99%

Descriptive Prompt Paraphrasing for Target-Oriented Multimodal Sentiment Classification

Liu,

Li,

Tao

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Target-Oriented Multimodal Sentiment Classification (TMSC) aims to perform sentiment polarity on a target jointly considering its corresponding multiple modalities including text, image, and others. Current researches mainly work on either of two types of targets in a decentralized manner. One type is entity, such as a person name, a location name, etc. and the other is aspect, such as 'food', 'service', etc. We believe that this target type based division in task modelling is not necessary because the sentiment polarity of the specific target is not governed by its type but its context. For this reason, we propose a unified model for targetoriented multimodal sentiment classification, so called UnifiedTMSC. It is prompt-based language modelling and performs well on four datasets spanning the above two target types. Specifically, we design descriptive prompt paraphrasing to reformulate TMSC task via (1) task paraphrasing, which obtains paraphrased prompts based on the task description through a paraphrasing rule, and (2) image prefix tuning, which optimizes a small continuous image vector throughout the multimodal representation space of text and images. Conducted on two entity-level multimodal datasets: Twitter-2015 and Twitter-2017, and two aspect-level multimodal datasets: Multi-ZOL and MASAD, the experimental results show the effectiveness of our UnifiedTMSC.

show abstract

Section: Introductionmentioning

confidence: 99%