Multimodal Aspect Extraction with Region-Aware Alignment Network

Wu, Hanqian; Cheng, Siliang; Wang, Jingjing; Li, Shoushan; Chi, Lian

doi:10.1007/978-3-030-60450-9_12

Cited by 21 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since contents in different modalities are often closely related, exploiting such multimodal information can help better analyze users' sentiments towards different aspects. Recent studies on multimodal ABSA mainly concentrate on simple ABSA tasks such as multimodal ATE [182,183] and multimodal ASC [184,185,186,187]. To align the information from different modalities, the text and image are often first encoded to feature representations, then some interaction networks are designed to fuse the information for making the final prediction.…”

Section: Multimodal Absamentioning

confidence: 99%

A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges

Zhang¹,

Li²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

As an important fine-grained sentiment analysis problem, aspect-based sentiment analysis (ABSA), aiming to analyze and understand people's opinions at the aspect level, has been attracting considerable interest in the last decade. To handle ABSA in different scenarios, various tasks have been introduced for analyzing different sentiment elements and their relations, including the aspect term, aspect category, opinion term, and sentiment polarity. Unlike early ABSA works focusing on a single sentiment element, many compound ABSA tasks involving multiple elements have been studied in recent years for capturing more complete aspect-level sentiment information. However, a systematic review of various ABSA tasks and their corresponding solutions is still lacking, which we aim to fill in this survey. More specifically, we provide a new taxonomy for ABSA which organizes existing studies from the axes of concerned sentiment elements, with an emphasis on recent advances of compound ABSA tasks. From the perspective of solutions, we summarize the utilization of pre-trained language models for ABSA, which improved the performance of ABSA to a new stage. Besides, techniques for building more practical ABSA systems in cross-domain/lingual scenarios are discussed. Finally, we review some emerging topics and discuss some open challenges to outlook potential future directions of ABSA.

show abstract

Section: Multimodal Absamentioning

confidence: 99%

A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges

Zhang¹,

Li²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Approaches for Multimodal Aspect Term Extraction (MATE). 1) RAN (Wu et al, 2020a), which aligns text with object regions by a coattention network. 2) UMT , which uses Cross-Modal Transformer to fuse text and image representations for Multimodal Named Entity Recognition (MNER).…”

Section: Compared Systemsmentioning

confidence: 99%

Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis

Liu¹,

Yu²,

Xia³

2022

Preprint

View full text Add to dashboard Cite

As an important task in sentiment analysis, Multimodal Aspect-Based Sentiment Analysis (MABSA) has attracted increasing attention in recent years. However, previous approaches either (i) use separately pre-trained visual and textual models, which ignore the crossmodal alignment or (ii) use vision-language models pre-trained with general pre-training tasks, which are inadequate to identify finegrained aspects, opinions, and their alignments across modalities. To tackle these limitations, we propose a task-specific Vision-Language Pre-training framework for MABSA (VLP-MABSA), which is a unified multimodal encoder-decoder architecture for all the pretraining and downstream tasks. We further design three types of task-specific pre-training tasks from the language, vision, and multimodal modalities, respectively. Experimental results show that our approach generally outperforms the state-of-the-art approaches on three MABSA subtasks. Further analysis demonstrates the effectiveness of each pretraining task. The source code is publicly released at https://github.com/NUSTM/ VLP-MABSA.

show abstract

“…The first group are the most related approaches to multi-modal aspect terms extraction. 1) RAN (Wu et al, 2020a); a co-attention approach for aspect terms extraction in a multi-modal scenario. 2) UMT (Yu et al, 2020b); 3) OSCGA (Wu et al, 2020b), an NER approach in a multi-modal scenario based on object features with BIO tagging.…”

Section: Baselinesmentioning

confidence: 99%

“…Previous studies normally cast MALSA in social media as two independent sub-tasks: Multimodal Aspect Terms Extraction (MATE) and Multimodal Aspect Sentiment Classification (MASC). First, MATE aims to detect a set of all potential * Corresponding Author aspect terms from a free text with its accompanying image (Wu et al, 2020a). Second, MASC aims to classify the sentiment polarity of a multi-modal post towards a given aspect in textual modality (Yu and Jiang, 2019).…”

Section: Introductionmentioning

confidence: 99%

Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal Relation Detection

Ju¹,

Zhang²,

Xiao³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

Aspect terms extraction (ATE) and aspect sentiment classification (ASC) are two fundamental and fine-grained sub-tasks in aspect-level sentiment analysis (ALSA). In the textual analysis, jointly extracting both aspect terms and sentiment polarities has been drawn much attention due to the better applications than individual sub-task. However, in the multimodal scenario, the existing studies are limited to handle each sub-task independently, which fails to model the innate connection between the above two objectives and ignores the better applications. Therefore, in this paper, we are the first to jointly perform multi-modal ATE (MATE) and multi-modal ASC (MASC), and we propose a multi-modal joint learning approach with auxiliary cross-modal relation detection for multi-modal aspect-level sentiment analysis (MALSA). Specifically, we first build an auxiliary text-image relation detection module to control the proper exploitation of visual information. Second, we adopt the hierarchical framework to bridge the multi-modal connection between MATE and MASC, as well as separately visual guiding for each sub module. Finally, we can obtain all aspect-level sentiment polarities dependent on the jointly extracted specific aspects. Extensive experiments show the effectiveness of our approach against the joint textual approaches, pipeline and collapsed multi-modal approaches.

show abstract

Multimodal Aspect Extraction with Region-Aware Alignment Network

Cited by 21 publications

References 17 publications

A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges

A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges

Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis

Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal Relation Detection

Contact Info

Product

Resources

About