Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictions

Abstract

Modern Review Helpfulness Prediction systems are dependent upon multiple modalities, typically texts and images. Unfortunately, those contemporary approaches pay scarce attention to polish representations of crossmodal relations and tend to suffer from inferior optimization. This might cause harm to model's predictions in numerous cases. To overcome the aforementioned issues, we propose Multi-modal Contrastive Learning for Multimodal Review Helpfulness Prediction (MRHP) problem, concentrating on mutual information between input modalities to explicitly elaborate cross-modal relations. In addition, we introduce Adaptive Weighting scheme for our contrastive learning approach in order to increase flexibility in optimization. Lastly, we propose Multimodal Interaction module to address the unalignment nature of multimodal data, thereby assisting the model in producing more reasonable multimodal representations. Experimental results show that our method outperforms prior baselines and achieves stateof-the-art results on two publicly available benchmark datasets for MRHP problem.

Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment

“…Advanced machine learning algorithms such as the Contrastive Learning framework have been applied to the natural language processing and computer vision [31,33,34,36,37,45]. Optimal Transport has also been extensively used in many natural language processing tasks and also the integration of vision and language fields, for example, Cross-Lingual Abstractive Summarization [32], machine translation [6], Vision and language pretraining [7,17], Visual Question Answering [5], etc.…”

InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling

“…In practice, mutual information maximization is approximated with a tractable lower bound, such as InfoNCE (Van den Oord, Li, and Vinyals 2018) and InfoMax (Hjelm et al 2019). These are also known as contrastive learning (Arora et al 2019;Wang and Isola 2020;Nguyen et al 2022;) that learns the representation similarity of positive and negative samples. Some recent studies (Xu et al 2022) apply mutual information for monolingual topic modeling and focus on the representations of documents.…”