Utilizing BERT Intermediate Layers for Multimodal Sentiment Analysis

Zou, Wenwen; Ding, Jianli; Wang, Chao

doi:10.1109/icme52920.2022.9860014

Cited by 9 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [115] propose a disentangled sentiment representation adversarial network (DiSRAN) to reduce the domain shift of expressive styles for cross-domain sentiment analysis. Recent works [116], [117], [118], [119], [120] tend to focus on disentangling the rich information among multi modalities and leveraging that to perform various downstream tasks. Alaniz et al [116] propose to use the semantic structure of the text to disentangle the visual data, in order to learn an unified representation between the text and image.…”

Section: Multimodal Applicationmentioning

confidence: 99%

“…Multimodal Application [113], [115] VAE-based Supervised [118] GAN-based [117], [119], [120] Others [26], [28], [123], [124], [125] VAE-based Supervised Recommendation [27], [29], [122] Others [128], [129], [130], [131] VAE-based Supervised Graph [29], [30], [132] Others tation z, which is fine-grained and ii) vector-wise: use two or more independent vectors z 1 , z 2 ... to represent different parts of data features, which is coarse-grained. To guarantee the disentanglement property, approach i) usually requires that z is dimension-wise independent, while approach ii) usually requires that z i is independent with z j where i = j.…”

Section: Papersmentioning

confidence: 99%

See 1 more Smart Citation

Disentangled Representation Learning for Recommendation

Wang

Chen

Zhou

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Disentangled Representation Learning (DRL) aims to learn a model capable of identifying and disentangling the underlying factors hidden in the observable data in representation form. The process of separating underlying factors of variation into variables with semantic meaning benefits in learning explainable representations of data, which imitates the meaningful understanding process of humans when observing an object or relation. As a general learning strategy, DRL has demonstrated its power in improving the model explainability, controlability, robustness, as well as generalization capacity in a wide range of scenarios such as computer vision, natural language processing, data mining etc. In this article, we comprehensively review DRL from various aspects including motivations, definitions, methodologies, evaluations, applications and model designs. We discuss works on DRL based on two well-recognized definitions, i.e., Intuitive Definition and Group Theory Definition. We further categorize the methodologies for DRL into four groups, i.e., Traditional Statistical Approaches, Variational Auto-encoder Based Approaches, Generative Adversarial Networks Based Approaches, Hierarchical Approaches and Other Approaches. We also analyze principles to design different DRL models that may benefit different tasks in practical applications. Finally, we point out challenges in DRL as well as potential research directions deserving future investigations. We believe this work may provide insights for promoting the DRL research in the community.

show abstract

Section: Multimodal Applicationmentioning

confidence: 99%

Section: Papersmentioning

confidence: 99%

Disentangled Representation Learning for Recommendation

Wang

Chen

Zhou

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…To increase the precision and viability of the evaluation of troop equipment, they suggested a sentiment analysis model based on attention and contextual factors for extracting essential insights from soldiers' remarks on equipment [3]. Wenwen Zou and others proposed an external hierarchical fusion structure that fuses BERT middle layer information with non-linguistic modal multi-stage fusion for BERT fine-tuning of multimodal linguistic data on CMU-MOSI and CMU-MOSEI datasets [4]. Yusuf Ziya Poyraz explained how BERT was utilized to analyze Sindhi newspaper datasets, which addressed the difficulties in Sindhi language analysis by improving accuracy through transliteration and lexical addition [5].…”

Section: Introductionmentioning

confidence: 99%

Identifying positive and negative comments underneath technology videos based on Bert's model

Lu,

Qiu,

Yang

2024

ACE

View full text Add to dashboard Cite

With the gradual popularization of the Internet and the increasing richness of content, more and more popular science videos appear in peoples vision. However, people have different attitudes towards them and cannot make explicit judgments. This paper used comments from some popular science videos on Bilibili as a dataset, trained the Bert model using parallel training methods such as (Distributed Data parallel) DDP, and selected different proportions of datasets (20%, 30%, and 40%) for training and testing data to objectively determine the positive and negative attitudes of people in comments. Research has found that different scale datasets have similar performance, with an 86% -87% accuracy rate. Besides, multi-card parallel processing can bring higher returns. The results help people comprehensively understand audience feedback and emotional preferences towards technical content. At the same time, it reveals the complex relationship between the emotions reflected in comments and various factors such as audience engagement, content strategies, and demographics.

show abstract

“…Further, they can be used to generate novel examples not found in the original dataset [ 21 ]. Such feature learning also supports a variety of other applications, such as super-resolution [ 22 ], multimodal application [ 23 , 24 , 25 , 26 , 27 ], medical imaging [ 28 , 29 ], video prediction [ 30 , 31 , 32 , 33 , 34 ], natural language processing [ 35 , 36 , 37 ], transfer learning and zero-shot learning [ 38 ].…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Learning of Disentangled Representation via Auto-Encoding: A Survey

Eddahmani¹,

Pham

Napoléon³

et al. 2023

Sensors

View full text Add to dashboard Cite

In recent years, the rapid development of deep learning approaches has paved the way to explore the underlying factors that explain the data. In particular, several methods have been proposed to learn to identify and disentangle these underlying explanatory factors in order to improve the learning process and model generalization. However, extracting this representation with little or no supervision remains a key challenge in machine learning. In this paper, we provide a theoretical outlook on recent advances in the field of unsupervised representation learning with a focus on auto-encoding-based approaches and on the most well-known supervised disentanglement metrics. We cover the current state-of-the-art methods for learning disentangled representation in an unsupervised manner while pointing out the connection between each method and its added value on disentanglement. Further, we discuss how to quantify disentanglement and present an in-depth analysis of associated metrics. We conclude by carrying out a comparative evaluation of these metrics according to three criteria, (i) modularity, (ii) compactness and (iii) informativeness. Finally, we show that only the Mutual Information Gap score (MIG) meets all three criteria.

show abstract

Utilizing BERT Intermediate Layers for Multimodal Sentiment Analysis

Cited by 9 publications

References 9 publications

Disentangled Representation Learning for Recommendation

Disentangled Representation Learning for Recommendation

Identifying positive and negative comments underneath technology videos based on Bert's model

Unsupervised Learning of Disentangled Representation via Auto-Encoding: A Survey

Contact Info

Product

Resources

About