Aspect-Level Sentiment Analysis Using CNN Over BERT-GCN

Abstract

Context-based GCNs have achieved relatively good effectiveness in the sentiment analysis task, especially aspect-level sentiment analysis (ALSA). However, the previous context-based GCNs for ALSA often used GCNs with the following limitations: (i) Using GCNs limited to a few layers (two or three) due to the vanishing gradient, limiting their performance. (ii) Not considering helpful information about the hidden context between the words. To solve these limitations, this paper proposes a novel CNN over the BERT-GCN model for ALSA. The contributions of the proposed method are summarized as follows: (i) Handling the disadvantage of limiting the GCN to a few layers by adding convolutional layers of the convolutional neural network (CNN) model after GCN layers. (ii) Considering further helpful information about the hidden context between the words by combining the Bidirectional Encoder Representations from Transformers (BERT) and Bidirectional Long Short Term Memory (BiLSTM) models. The proposed model includes the following steps: First, words in sentences are converted vectors using BERT. Second, the contextualized word representations are created based on BiLSTM over word vectors. Third, significant features are extracted and represented using the GCN model with multiple convolutional layers over the contextualized word representations. Finally, the aspect-level sentiments are classified using the CNN model over the feature vectors. Experiments on three benchmark datasets illustrate that our proposed model has improved the performance of the previous context-based GCN methods for ALSA.

Dense Concatenation Memory Network for Aspect Level Sentiment Analysis

“…The model includes self-supervised domain-specific fine-tuning of the BERT model and a crossdomain evaluation. 10) CNN over BERT-GCN [15]: This model adds a convolutional layer after the GCN layer. In this way, CNN can be used to analyze the important features in the sentence and the aspect emotion in the input sentence.…”

“…Most of these models use the attention mechanism and skip connection of ResNet to deepen the depth of the network, but this method may hinder the information flow in the network, resulting in the model can not make full use of the extracted features of each layer. CNN over BERT-GCN [15] proposed to use the combination of BERT and BiLSTM to take into account the useful information hidden in context, and exploit the GCN model with multiple convolutional layers to capture the context features. ASBABM [16] proposed an aspect-level sentiment analysis model based on BERT and multi-level attention mechanisms.…”

Dependency parsing masking and BERT for sentiment analysis

“…In recent years, deep learning models, such as Convolutional Neural Network (CNN) [10] , Long Short-Term Memory (LSTM) [11] , Gated Recurrent Unit (GRU) [12] , Attention mechanism (Attention) [13] , etc., have achieved better results in the field of sentiment analysis. These models can effectively use a large amount of text data to capture contextual and emotional information, thus improving the accuracy of sentiment analysis.…”

“…The model uses only sentence sequences as the input to the BERT encoder, and proposes and uses a new interactive gate mechanism, which becomes common gate, to effectively reduce the interference of noisy words, obtain contextual information, and enhance the sentiment prediction effect on aspects. PHAN et al [10] proposed a new BERT-GCN model that combines the BERT and BILSTM models to address the limitations of context-based graph convolutional networks for aspect-based sentiment analysis tasks. By adding convolutional layers to the convolutional neural network (CNN) model, the model can handle the drawback of GCN being limited to only a few layers and further improve the performance by considering the beneficial information of hidden context between words.…”

“…In this module, the output of the BERT model is multiplied with the masking matrix to fuse the contextual information with the dependency structure information, the calculation formula is shown in equation (10).…”