Attention-Based Convolutional Neural Networks for Sentence Classification

Zhao, Zhiwei; Wu, Youzheng

doi:10.21437/interspeech.2016-354

Cited by 88 publications

(44 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• ATT-CNN [43]: It uses attention mechanism to capture long-term dependence information and correlation between nonconsecutive words automatically and then sents them to CNN. The parameters are the same with [10] in CNN.…”

Section: B Baseline Methodsmentioning

confidence: 99%

Multi-Channel CNN Based Inner-Attention for Compound Sentence Relation Classification

Sun

Deng

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Relation classification is a vital task in natural language processing, and it is screening for semantic relation between clauses in texts. This paper describes a study of relation classification on Chinese compound sentences without connectives. There exists an implicit relation in a compound sentence without connectives, which makes it difficult to realize the recognition of relation. The major challenges that relation classification modeling faces are how to obtain the contextual representation of sentence and relation dependence features between clauses. To solve this problem, we propose a novel Inatt-MCNN model to extract sentence features and classify relations by combining multi-channel CNN and Inner-attention mechanism. This network structure utilizes CNN to extract local features of sentences and Inner-attention to capture sentence-level feature representations for this relation classification task. Besides, since the Innerattention is based on Bi-LSTM, the global and long-term dependence semantic information can be well obtained in Inatt-MCNN to promote the model performance. We conduct experiments on two public Chinese discourse datasets: the Chinese compound sentence corpus (CCCS) dataset and the Tsinghua Chinese Treebank(TCT) dataset. Compared with the previous public methods, Inatt-MCNN model has superior performance and achieves the highest accuracy, especially on the CCCS dataset.INDEX TERMS Relation classification, multi-channel CNN, inner-attention mechanism, Chinese compound sentence without connectives.

show abstract

Section: B Baseline Methodsmentioning

confidence: 99%

Multi-Channel CNN Based Inner-Attention for Compound Sentence Relation Classification

Sun

Deng

et al. 2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The probability module calculates the activation probability of the capsule according to the semantic feature , combined with formula (18).…”

Section: Figure 6 Capsule Structure Diagrammentioning

confidence: 99%

“…The attention mechanism can achieve selective focus on important information. Zhao et al [18] proposed an ATT-CNN model combining attention mechanism and CNN to effectively identify the importance of words in a sentence. Vaswani et al [19] proposed multi-head attention mechanism adopted in the transformer translation model allows the model to obtain more levels of information in sentences from different spaces and improve the feature expression ability of the model.…”

Section: Introductionmentioning

confidence: 99%

Sentiment Analysis Using Multi-Head Attention Capsules With Multi-Channel CNN and Bidirectional GRU

Chen

Sun

Chen³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Existing text sentiment analysis methods mostly rely on a large number of language knowledge and sentiment resources. This paper proposes the Multi-channel convolution and bidirectional GRU multi-head attention capsule(AT-MC-BiGRU-Capsule), which uses vector neurons to replace scalar neurons to model text emotions, and uses capsules to characterize text emotions. In addition, traditional methods cannot extract the multi-level features of text sequence well. Multi-head attention can encode the dependencies between words, capture sentiment words in text, and using Convolutional Neural Network(CNN) and Bidirectional gated recurrent unit network(Bi-GRU) to extract local features and global semantic features of text respectively, the global average pooling layer is introduced to obtain the multilevel feature representation of the text sequence more comprehensively. This paper selects three English datasets and one Chinese dataset in the general corpus of sentiment classification to conduct experiments, and achieves better results than other baseline models.

show abstract

“…The parameter setting in convolution neural network for all above models refer to Kim's paper, the convolution filter widths H are set to [2,3,4]. Each width has a set of 100 convolution filters.We conduct grid search on MR datasets and find that Model III can get a good performance when the hidden state dimension k in Bi-LSTM is set to 100, the dimension of Bi-LSTM and the global feature selection vector is set to 100 during training.…”

Section: 2baselines and Parametersmentioning

confidence: 99%

A Hybrid Neural Network for Sentence Classification

Zhou¹,

Du²

2017

Proceedings of the 7th International Conference on Computer Engineering and Networks — PoS(CENet2017)

View full text Add to dashboard Cite

The sentence classification is the foundation of many Natural Language Processing applications. Prior neural network which use one type network for sentence classification can't use the abundant information in a sentence. In this paper, we proposed a hybrid neural network in combination with recurrent neural network and convolutional neural networks for sentence classification. The recurrent neural network can model long distance global information in a text, but it can't effectively extract the local information and convolutional neural network inversely. The proposed hybrid neural network takes full advantage of the advantages of these two networks while extracting global feature and local feature at the same time. In order to get the global feature, we also proposed three different methods to make use of hidden states generated by recurrent neural network. We conducted experiments on four public open datasets. The results show that our hybrid neural network does better than models by using the recurrent neural network or the convolutional neural networks alone, higher and completive classification accuracy is obtained.

show abstract

Attention-Based Convolutional Neural Networks for Sentence Classification

Cited by 88 publications

References 13 publications

Multi-Channel CNN Based Inner-Attention for Compound Sentence Relation Classification

Multi-Channel CNN Based Inner-Attention for Compound Sentence Relation Classification

Sentiment Analysis Using Multi-Head Attention Capsules With Multi-Channel CNN and Bidirectional GRU

A Hybrid Neural Network for Sentence Classification

Contact Info

Product

Resources

About