Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction

Nguyen, T. Q.; Wu, Xiaobao; Dong, Xinshuai; Cong-Duy, Nguyen,; Luu, Anh Tuan

doi:10.18653/v1/2023.findings-acl.106

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Advanced machine learning algorithms such as the Contrastive Learning framework have been applied to the natural language processing and computer vision [31,33,34,36,37,45]. Optimal Transport has also been extensively used in many natural language processing tasks and also the integration of vision and language fields, for example, Cross-Lingual Abstractive Summarization [32], machine translation [6], Vision and language pretraining [7,17], Visual Question Answering [5], etc.…”

Section: Related Workmentioning

confidence: 99%

Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment

Nguyen,

Vu-Le,

Nguyen

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

Language models have been supervised with both language-only objective and visual grounding in existing studies of visual-grounded language learning. However, due to differences in the distribution and scale of visual-grounded datasets and language corpora, the language model tends to mix up the context of the tokens that occurred in the grounded data with those that do not. As a result, during representation learning, there is a mismatch between the visual information and the contextual meaning of the sentence. To overcome this limitation, we propose GroundedBERT -a grounded language learning method that enhances the BERT representation with visually grounded information. GroundedBERT comprises two components: (i) the original BERT which captures the contextual representation of words learned from the language corpora, and (ii) a visual grounding module which captures visual information learned from visual-grounded datasets. Moreover, we employ Optimal Transport (OT), specifically its partial variant, to solve the fractional alignment problem between the two modalities. Our proposed method significantly outperforms the baseline language models on various language tasks of the GLUE and SQuAD datasets. CCS CONCEPTS• Computing methodologies → Natural language processing; Computer vision; • Theory of computation → Theory and algorithms for application domains.

show abstract