Initializing Convolutional Filters with Semantic Features for Text
            Classification

Shen, Li; Zhao, Zhe; Liu, Tao; Hu, Renfen; Du, Xiaoyong

doi:10.18653/v1/d17-1201

Cited by 50 publications

(28 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They supposed words derived from an external resource, which can be divided into N group. And this is similar to Li's parameter initialization through clustering [29]. Our model is consistent with its purpose.…”

Section: Related Worksupporting

confidence: 69%

“…Johnson et al [28] proposed a deep pyramid CNNs, they utilized the downsampling to decrease computational cost while efficiently representing long range associations in text. Li et al [29] employ initializing convolutional filters with features that computed by K-means rather than initializing filters randomly, which enable features to be efficiently captured. Zhang et al [30] proposed a grouped weight sharing way to instead of word embeddings.…”

Section: Related Workmentioning

confidence: 99%

“…For these CNN guided sentence models [13,[26][27][28][29][30], the performance is limited by the size of the perception field, thus it is less effective in capturing the context of the sentence than the RNNs. In this work, one insight is that the implicit semantics of the sentence is often highly context-dependent, thus we utilize RNN to encode sentences which can better capture the context relationship, so that the model pays more attention to the most important information in the sentence.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification

Zhang¹,

Hong²,

Han³

et al. 2019

IJCIS

View full text Add to dashboard Cite

Recurrent neural network (RNN) is one of the most popular architectures for addressing variable sequence text, and it shows outstanding results in many natural language processing (NLP) tasks and remarkable performance in capturing long-term dependencies. Many models have achieved excellent results based on RNN. However, most of these models overlook the locations of the keywords in a sentence and the semantic connections in different directions. As a consequence, these methods do not make full use of the available information. Considering that different words in a sequence usually have different importance, in this paper, we propose bidirectional gated recurrent units (BGRUs) which integrates a novel attention pooling mechanism with max-pooling operation to force the model to pay attention to the keywords in a sentence and maintain the most meaningful information of the text automatically. The presented model allows to encode longer sequences. Thus, it not only prevents important information from being discarded but also can be used to filter noises. To avoid full exposure of content without any control, we add an output gate to the GRU, which is named as text unit. The proposed model was evaluated on multiple tasks, including sentimental classification, movie review data, and a subjective classification dataset. Experimental results show that our model can achieve excellent performance on these tasks.

show abstract

Section: Related Worksupporting

confidence: 69%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification

Zhang¹,

Hong²,

Han³

et al. 2019

IJCIS

View full text Add to dashboard Cite

show abstract

“…We compare our models with several state-of-the-art baseline models, including Bow+SVM (Chris Manning et al 2012), CNN (Kim 2014), CharCNN (Zhang, Zhao, and Le-Cun 2015), CNN-non-static+UNI (Li et al 2017), KPCNN (Wang et al 2017).…”

Section: Baseline Methodsmentioning

confidence: 99%

Incorporating Context-Relevant Knowledge into Convolutional Neural Networks for Short Text Classification

Cai

2019

AAAI

View full text Add to dashboard Cite

Some text classification methods don’t work well on short texts due to the data sparsity. What’s more, they don’t fully exploit context-relevant knowledge. In order to tackle these problems, we propose a neural network to incorporate context-relevant knowledge into a convolutional neural network for short text classification. Our model consists of two modules. The first module utilizes two layers to extract concept and context features respectively and then employs an attention layer to extract those context-relevant concepts. The second module utilizes a convolutional neural network to extract high-level features from the word and the contextrelevant concept features. The experimental results on three datasets show that our proposed model outperforms the stateof-the-art models.

show abstract

“…In this process, the n-grams are extracted from the train data and clustered by k-means. The experimental results show that the method effectively reduces the error caused by parameter adjustment [11]. Johnson et al propose a deep pyramid convolutional neural network model, which improves the classification accuracy by deepening the hierarchical structure of convolutional neural networks.…”

Section: Text Classification In General Domainmentioning

confidence: 99%

A Novel Neural Network-Based Method for Medical Text Classification

Weng

Ding

2019

Future Internet

View full text Add to dashboard Cite

Medical text categorization is a specific area of text categorization. Classification for medical texts is considered a special case of text classification. Medical text includes medical records and medical literature, both of which are important clinical information resources. However, medical text contains complex medical vocabularies, medical measures, which has problems with high-dimensionality and data sparsity, so text classification in the medical domain is more challenging than those in other general domains. In order to solve these problems, this paper proposes a unified neural network method. In the sentence representation, the convolutional layer extracts features from the sentence and a bidirectional gated recurrent unit (BIGRU) is used to access both the preceding and succeeding sentence features. An attention mechanism is employed to obtain the sentence representation with the important word weights. In the document representation, the method uses the BIGRU to encode the sentences, which is obtained in sentence representation and then decode it through the attention mechanism to get the document representation with important sentence weights. Finally, a category of medical text is obtained through a classifier. Experimental verifications are conducted on four medical text datasets, including two medical record datasets and two medical literature datasets. The results clearly show that our method is effective.

show abstract

Initializing Convolutional Filters with Semantic Features for Text Classification

Cited by 50 publications

References 21 publications

Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification

Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification

Incorporating Context-Relevant Knowledge into Convolutional Neural Networks for Short Text Classification

A Novel Neural Network-Based Method for Medical Text Classification

Contact Info

Product

Resources

About