Cold-Start Aware User and Product Attention for Sentiment Classification

Abstract

The use of user/product information in sentiment analysis is important, especially for cold-start users/products, whose number of reviews are very limited. However, current models do not deal with the cold-start problem which is typical in review websites. In this paper, we present Hybrid Contextualized Sentiment Classifier (HCSC), which contains two modules: (1) a fast word encoder that returns word vectors embedded with short and long range dependency features; and (2) Cold-Start Aware Attention (CSAA), an attention mechanism that considers the existence of cold-start problem when attentively pooling the encoded word vectors. HCSC introduces shared vectors that are constructed from similar users/products, and are used when the original distinct vectors do not have sufficient information (i.e. cold-start). This is decided by a frequency-guided selective gate vector. Our experiments show that in terms of RMSE, HCSC performs significantly better when compared with on famous datasets, despite having less complexity, and thus can be trained much faster. More importantly, our model performs significantly better than previous models when the training data is sparse and has coldstart problems.

Sentiment analysis using deep learning approaches: an overview

“…An individual customer first checks the ratings of a product and opinions from other customers before making his/her purchasing decision, and business organizations use sentiment analysis tools to understand their customers' feeling. Several efforts [11][12][13][14] have been made in applying sentiment analysis to customer reviews. Therefore, analyzing customer reviews have been proved to boost the relevant market and increase the confidence of customers [15].…”

Categorical Metadata Representation for Customized Text Classification

“…We investigate at how basis vectors understand word-level semantics through the lens of the attention vectors they create. Previous models either combine user/product information into a single attention vector (Chen et al, 2016) or entirely separate them into distinct user and product attention vectors (Amplayo et al, 2018a). On the other hand, our model creates a single attention vector, but through the k basis attention vectors, which are vectors containing fuzzy semantics among users and products.…”

“…Previous literature only focused on the analysis (Amplayo et al, 2018a) and case studies (Chen et al, 2016) of word-level customized dependencies, usually through attention vectors. In this paper, we additionally investigate the documentlevel customized dependencies, i.e., how our basis-customization changes the document-level semantics when a category is different.…”

Rethinking Attribute Representation and Injection for Sentiment Classification

“…Prior to the deep learning era, these information were used as effective categorical features Tan et al, 2011;Gao et al, 2013;Park et al, 2015) for the machine learning model. Recent work has used them to improve the overall performance (Chen et al, 2016;Dong et al, 2017), interpretability (Amplayo et al, 2018a;Angelidis and Lapata, 2018), and personalization (Ficler and Goldberg, 2017) of neural network models in different tasks such as sentiment classification (Tang et al, 2015), review summarization (Yang et al, 2018a), and text generation (Dong et al, 2017).…”

“…where u and p are the user and product embeddings, and h is a word encoding from BiLSTM. Since then, most of the subsequent work attempted to improve the model by extending the model architecture to be able to utilize external features (Zhu and Yang, 2017), handle cold-start entities (Amplayo et al, 2018a), and represent user and product separately (Ma et al, 2017). Intuitively, however, this method is not the ideal method to represent and inject attributes because of two reasons.…”