ConFEDE: Contrastive Feature Decomposition for Multimodal Sentiment Analysis

Yang, Jiuding; Yu, Yakun; Niu, Di; Guo, Weidong; Xu, Yan

doi:10.18653/v1/2023.acl-long.421

Cited by 34 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(4) Self-MM : Generate unimodal labels through a designed self-supervised learning strategy, and train both unimodal and jointly to learn consistency and differences between modalities. (5) ConFEDE (Yang et al, 2023): Interactions between modalities are achieved by modeling specific view interactions and cross-view interactions, and they are fused in the temporal dimension through multi-view gating mechanisms.…”

Section: Comparative On the Ch-sims Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Sentiment Analysis Method Based on Hierarchical Adaptive Feature Fusion Network

Zhang

2024

International Journal on Semantic Web and Information Systems

View full text Add to dashboard Cite

The traditional multi-modal sentiment analysis (MSA) method usually considers the multi-modal characteristics to be equally important and ignores the contribution of different modes to the final MSA result. Therefore, an MSA method based on hierarchical adaptive feature fusion network is proposed. Firstly, RoBERTa, ResViT, and LibROSA are used to extract different modal features and construct a layered adaptive multi-modal fusion network. Then, the multi-modal feature extraction module and cross-modal feature interaction module are combined to realize the interactive fusion of information between modes. Finally, an adaptive gating mechanism is introduced to design a global multi-modal feature interaction module to learn the unique features of different modes. The experimental results on three public data sets show that the proposed method can make full use of multi-modal information, outperform other advanced comparison methods, improve the accuracy and robustness of sentiment analysis, and is expected to achieve better results in the field of sentiment analysis.

show abstract

Section: Comparative On the Ch-sims Datasetmentioning

confidence: 99%

“…The following is a comparative of the proposed model with TFN (Zadeh et al, 2017), MulT (Tsai et al, 2019), MISA (Hazarika et al, 2020), Self-MM , and ConFEDE (Yang et al, 2023) based on the CH-SIMS dataset:…”

Section: Comparative On the Ch-sims Datasetmentioning

confidence: 99%