Rethinking Attribute Representation and Injection for Sentiment Classification

Amplayo, Reinald Kim

doi:10.18653/v1/d19-1562

Cited by 17 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For better performance of document-level sentiment classification, some researchers have conducted studies to introduce user representations and item representations in the analysis of review texts [13]- [16]. The intuition is that these representations can provide global information such as rating and language preferences of users and overall ratings of items [23].…”

Section: B Personalized Sentiment Classificationmentioning

confidence: 99%

“…Specifically, they store representations of representative users or items in memory slots and then use them to infer representations for cold-start users or items. Different from all these methods, Amplayo et al [13] attempt to represent users and items with their proposed "chunk-wise" weight matrices instead of bias vectors and inject them into four locations (i.e. embedding, encoding, attention, classifier) in a model.…”

Section: B Personalized Sentiment Classificationmentioning

confidence: 99%

“…• HUAPA [15] uses two separate networks with the same structure to model the text from the view of user and item respectively and combine them for final prediction. • CHIM [13] studies how and where to incorporate user and item information in a sentiment classification model. • RRP-UPM [16] is based on HUAPA and considers the inherent correlation between users or items for better representation.…”

Section: B Baselinesmentioning

confidence: 99%

See 2 more Smart Citations

Injecting User Identity Into Pretrained Language Models for Document-Level Sentiment Classification

Cao

Zhuang

2022

IEEE Access

View full text Add to dashboard Cite

This paper mainly studies the combination of pre-trained language models and user identity information for document-level sentiment classification. In recent years, pre-trained language models (PLMs) such as BERT have achieved state-of-the-art results on many NLP applications, including document-level sentiment classification. On the other hand, a collection of works introduce additional information such as user identity for better text modeling. However, most of them inject user identity into traditional models, while few studies have been conducted to study the combination of pre-trained language models and user identity for even better performance. To address this issue, in this paper, we propose to unite user identity and PLMs and formulate User-enhanced Pre-trained Language Models (U-PLMs). Specifically, we demonstrate two simple yet effective attempts, i.e. embedding-based and attention-based personalization, which inject user identity into different parts of a pre-trained language model and provide personalization from different perspectives. Experiments in three datasets with two backbone PLMs show that our proposed methods outperform the best state-of-the-art baseline method with an absolute improvement of up to 3%, 2.8%, and 2.2% on accuracy. In addition, our methods encode user identity with plugin modules, which are fully compatible with most auto-encoding pre-trained language models.

show abstract

Section: B Personalized Sentiment Classificationmentioning

confidence: 99%

Section: B Personalized Sentiment Classificationmentioning

confidence: 99%

Section: B Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

Injecting User Identity Into Pretrained Language Models for Document-Level Sentiment Classification

Cao

Zhuang

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Wang et al (2018) proposed an adversarial cross-lingual learning framework to utilize both English and Chinese corpora for personalized microblog sentiment analysis. Amplayo (2019) investigated the influences of different ways and locations of the attribute incorporation. However, the aforementioned approaches did not consider the data sparsity issue that most users only generate limited content online.…”

Section: Introductionmentioning

confidence: 99%

A Neural Group-wise Sentiment Analysis Model with Data Sparsity Awareness

Zhou¹,

Zhang²,

Zhang³

et al. 2021

AAAI

View full text Add to dashboard Cite

Sentiment analysis on user-generated content has achieved notable progress by introducing user information to consider each individual’s preference and language usage. However, most existing approaches ignore the data sparsity problem, where the content of some users is limited and the model fails to capture discriminative features of users. To address this issue, we hypothesize that users could be grouped together based on their rating biases as well as degree of rating consistency and the knowledge learned from groups could be employed to analyze the users with limited data. Therefore, in this paper, a neural group-wise sentiment analysis model with data sparsity awareness is proposed. The user-centred document representations are generated by incorporating a group-based user encoder. Furthermore, a multi-task learning framework is employed to jointly modelusers’ rating biases and their degree of rating consistency. One task is vanilla populationlevel sentiment analysis and the other is groupwise sentiment analysis. Experimental results on three real-world datasets show that the proposed approach outperforms some state-of the-art methods. Moreover, model analysis and case study demonstrate its effectiveness of modeling user rating biases and variances.

show abstract

“…Later, inspired by the question-answering task, some work [16,57] further exploits the same idea with memory network [15] for a better document representation learning. Likewise, Amplayo [58] proposes a chunk-wise importance matrix (CHIM) based representation to represent the user and product information.…”

Section: Integrating User and Product Informationmentioning

confidence: 99%

Opinion mining on product reviews from document to clause

Lin¹

View full text Add to dashboard Cite

A thesis submitted to the Nanyang Technological University in partial fulfilment of the requirement for the degree of Doctor of Philosophy 2022 Sun Aixin. I am fortunate to meet Prof.Sun. Without his patience, kindness, guidance and encouragement, I could not finish this thesis or my PhD study. Through his tireless one-to-one instruction on my research problems, I learned Natural Language Processing knowledge. More importantly, I learned how to think critically about research problems. Also, I thank the TAC members, thesis examiners, and the oral defence panel for giving valuable suggestions to my research work.Furthermore, I want to thank all my friends from schools, labs and colleagues I met for their continuous encouragement and inspiration. In particular, I thank my brilliant labmates at NTU for sharing ideas and discussion: Yequan Wang, Xin Zheng, Grace Lee,

show abstract

Rethinking Attribute Representation and Injection for Sentiment Classification

Cited by 17 publications

References 36 publications

Injecting User Identity Into Pretrained Language Models for Document-Level Sentiment Classification

Injecting User Identity Into Pretrained Language Models for Document-Level Sentiment Classification

A Neural Group-wise Sentiment Analysis Model with Data Sparsity Awareness

Opinion mining on product reviews from document to clause

Contact Info

Product

Resources

About