Sentiment Analysis Using Multi-Head Attention Capsules With Multi-Channel CNN and Bidirectional GRU

Chen, Yan; Sun, Huan; Chen, Haomai; Li, Meng; Cai, Yingying; Cai, Zixing; Huang, Jing

doi:10.1109/access.2021.3073988

Cited by 29 publications

(16 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, RNN can consider the long-distance dependency within texts. However, original RNNs suffer from gradient dispersion and gradient disappearance, which affect the learning process [3]. To solve this problem, the long short term memory (LSTM) model has been used [36].…”

Section: Sentiment Analysismentioning

confidence: 99%

“…Nowadays, text sentiment analysis has become essential for many fields such as movie recommendation, e-commerce, and public opinion analysis [2]. For example, sentiment analysis aims to obtain the sentiment tendency of the person's opinions towards products, hot events, or any specific topic, which helps human decision-making [3]. Generally, researchers have explored three types of sentiment analysis approaches dictionary-based sentiment methods, machine learning-based sentiment methods, and deep learning-based sentiment methods.…”

Section: Introductionmentioning

confidence: 99%

“…In recent years, deep learning models have attracted the attention of many researchers to address the problem of feature extraction. They propose various deep learning-based methods for sentiment analysis, which achieved promising results compared to machine learning methods in sentiment association and sentiment classification [3,[9][10][11]. However, deep learning models face the difficulty of extracting more comprehensive sentimental and emotional features since a large amount of emotional information is not utilized.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Transformer-Based Graph Convolutional Network for Sentiment Analysis

et al. 2022

View full text Add to dashboard Cite

Sentiment Analysis is an essential research topic in the field of natural language processing (NLP) and has attracted the attention of many researchers in the last few years. Recently, deep neural network (DNN) models have been used for sentiment analysis tasks, achieving promising results. Although these models can analyze sequences of arbitrary length, utilizing them in the feature extraction layer of a DNN increases the dimensionality of the feature space. More recently, graph neural networks (GNNs) have achieved a promising performance in different NLP tasks. However, previous models cannot be transferred to a large corpus and neglect the heterogeneity of textual graphs. To overcome these difficulties, we propose a new Transformer-based graph convolutional network for heterogeneous graphs called Sentiment Transformer Graph Convolutional Network (ST-GCN). To the best of our knowledge, this is the first study to model the sentiment corpus as a heterogeneous graph and learn document and word embeddings using the proposed sentiment graph transformer neural network. In addition, our model offers an easy mechanism to fuse node positional information for graph datasets using Laplacian eigenvectors. Extensive experiments on four standard datasets show that our model outperforms the existing state-of-the-art models.

show abstract

Section: Sentiment Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transformer-Based Graph Convolutional Network for Sentiment Analysis

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Then, we combine the output of convolutional layers and middle-level features. Furthermore, the two-layers GRU [44] is used to learn the new combined features. GRU layer aims to find the sequence information of the music.…”

Section: B Crnn With Multiple Featuresmentioning

confidence: 99%

Singer Identification for Metaverse with Timbral and Middle-Level Perceptual Features

Zhang¹,

Wang²,

Cheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Metaverse is an interactive world that combines reality and virtuality, where participants can be virtual avatars. Anyone can hold a concert in a virtual concert hall, and users can quickly identify the real singer behind the virtual idol through the singer identification. Most singer identification methods are processed using the frame-level features. However, expect the singer's timbre, the music frame includes music information, such as melodiousness, rhythm, and tonal. It means the music information is noise for using frame-level features to identify the singers. In this paper, instead of only the frame-level features, we propose to use another two features that address this problem. Middle-level feature, which represents the music's melodiousness, rhythmic stability, and tonal stability, and is able to capture the perceptual features of music. The timbre feature, which is used in speaker identification, represents the singers' voice features. Furthermore, we propose a convolutional recurrent neural network (CRNN) to combine three features for singer identification. The model firstly fuses the frame-level feature and timbre feature and then combines middle-level features to the mix features. In experiments, the proposed method achieves comparable performance on an average F1 score of 0.81 on the benchmark dataset of Artist20, which significantly improves related works.

show abstract

“…TC (text categorization), as an important task in NLP (natural language processing) [2], refers to the automatic recognition of text categories based on text content by text classifier. In recent years, Chinese text classification has been studied and put into practical applications in various fields in China, such as sentiment analysis, movie reviews, stock market, and classification tasks for fault-related descriptions [3]. With the development of database in the field of rail transportation in China, TC is also applied in this field as well.…”

Section: Introductionmentioning

confidence: 99%

BERT-BiGRU Intelligent Classification of Metro On-Board Equipment Faults Based on Key Layer Fusion

Liu

Lin

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Metro train operation may result in massive and complex unstructured fault text data. To solve the problem of low classification accuracy and incomplete classification effect of unstructured fault data automatic classification task, a BERT-BiGRU fault text classification model based on key layer fusion is proposed. Firstly, the unstructured text data is processed into word vectors with location information in the word embedding layer and then input into the BERT layer. Based on the traditional 12-layer BERT model, the semantic information is fully obtained by encoding the two-way transformer encoder in layers 2, 4, 6, 8, and 12 for fusion and dimensionality reduction, which is then input into the BiGRU layer to extract the context information to obtain the high-level feature representation of the text. After that, the final classification results are output in the output layer through the full connection layer FC and softmax functions. This model is tested with other mainstream models on the fault text data of metro on-board equipment. The experiment results show that on the same data set, the F 1 -score of this model is 7%~8% higher than that of the current mainstream classification model, that is, about 1~2% higher than that of other PLMs, and the F 1 -score of this model is higher than that of BERT model with different transformer layers, reaching the highest value of 0.9272, and the convergence speed in the training process is fast.

show abstract

Sentiment Analysis Using Multi-Head Attention Capsules With Multi-Channel CNN and Bidirectional GRU

Cited by 29 publications

References 19 publications

Transformer-Based Graph Convolutional Network for Sentiment Analysis

Transformer-Based Graph Convolutional Network for Sentiment Analysis

Singer Identification for Metaverse with Timbral and Middle-Level Perceptual Features

BERT-BiGRU Intelligent Classification of Metro On-Board Equipment Faults Based on Key Layer Fusion

Contact Info

Product

Resources

About