LSTM-CRF Neural Network With Gated Self Attention for Chinese NER

Jin, Yanliang; Xie, Jinfei; Guo, Weisi; Luo, Chuanwen; Wu, Dijia; Wang, Rui

doi:10.1109/access.2019.2942433

Cited by 49 publications

(23 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some research extended ordinary NN with a highway channel to construct deeper trainable networks for NER tasks [35,36]. This optimization based on the gated mechanism is also designed to control information flowing across layers, which showed verifiable improvement on some previous works [37,38]. For example, Kim' experiments [35] with CharCNN and a highway network showed its more powerful generalization ability that greatly enhances the language models on many fundamental NLP tasks.…”

Section: B Dl-based Neural Network For Ner Tasksmentioning

confidence: 99%

MoGCN: Mixture of Gated Convolutional Neural Network for Named Entity Recognition of Chinese Historical Texts

Yan

Su²,

Wang³

2020

IEEE Access

View full text Add to dashboard Cite

Named Entity Recognition (NER) systems have been largely advanced by deep neural networks in the recent decade. However, the state-of-the-arts on NER have been less applied to Chinese historical texts due to the lack of standard corpora in Chinese historical domains and the difficulty of accessing a quality ancient corpus. This paper addresses the respective issues and proposes an efficient automatic processing solution for tackling NER of ancient Chinese data, including the implementation of data-driven tagging and an innovative end-to-end network namely "MoGCN" (Mixture of Gated Convolutional Neural Network). A corpus consisting of three genres of Chinese historical classics is generated by our tagging approach, which is experimented for uncovering the generalization ability of proposed model. The empirical analysis demonstrates that our proposed model achieves the best results with above 1.5% F1-score improvement over other sophisticated models in this dataset, where the experimental performance shows positive dependence on the quality of corpus. Furthermore, our model can perform much better on shorter entities especially for 2-charater ones, while many long-range entities can be only identified by our model based on our auxiliary attribute analysis. This work serves as a preliminary exploitation of NER for historical data, providing unique insights and reference values for similar tasks. Future work should be focused on more exploration about NER optimization on massive Chinese traditional texts with linguistic features and learning strategies. INDEX TERMS Named entity recognition, gated neural network, Chinese historical texts

show abstract

Section: B Dl-based Neural Network For Ner Tasksmentioning

confidence: 99%

MoGCN: Mixture of Gated Convolutional Neural Network for Named Entity Recognition of Chinese Historical Texts

Yan

Su²,

Wang³

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Ref. [22,23] achieved state-of-the-art performance on NER by using LSTM-CRF models, with characters being integrated into word representations. With the development of Chinese NER task, on the basis of using the neural network model, researchers have also made good achievements in this task [24].…”

Section: Related Workmentioning

confidence: 99%

POISE: Efficient Cross-Domain Chinese Named Entity Recognization via Transfer Learning

Sheng

Wumaier

2020

Symmetry

View full text Add to dashboard Cite

To improve the performance of deep learning methods in case of a lack of labeled data for entity annotation in entity recognition tasks, this study proposes transfer learning schemes that combine the character to be the word to convert low-resource data symmetry into high-resource data. We combine character embedding, word embedding, and the embedding of the label features using high- and low-resource data based on the BiLSTM-CRF model, and perform the feature-transfer and parameter-sharing tasks in two domains of the BiLSTM network to annotate with zero resources. Before transfer learning, we must first calculate the label similarity between two different domains and select the label features with large similarity for feature transfer mapping. All training parameters of the source domain in the model are shared during the BiLSTM network processing and CRF layer. In addition, we also use the method of combining characters and words to reduce the problem of word segmentation across domains and reduce the error rate in label mapping. The results of experiments show that in terms of the overall F1 score, the proposed model without supervision was superior by 9.76 percentage points to the general parametric shared transfer learning method, and by 9.08 and 12.38 percentage points, respectively, to two recent high–low resource learning methods. The proposed scheme improves performance in terms of transfer learning between the high- and low-resource data and can identify the predicted data in the target domain.

show abstract

“…In recent years, attention mechanism used to learn text classification [35], question answering [36], and named entity recognition [37]. The words in the sentence contain different levels of importance [38].…”

Section: (3) Sentence Level Attention Filter Layermentioning

confidence: 99%

Attention-Based LSTM with Filter Mechanism for Entity Relation Classification

Jin

Guo

2020

Symmetry

Self Cite

View full text Add to dashboard Cite

Relation classification is an important research area in the field of natural language processing (NLP), which aims to recognize the relationship between two tagged entities in a sentence. The noise caused by irrelevant words and the word distance between the tagged entities may affect the relation classification accuracy. In this paper, we present a novel model multi-head attention long short term memory (LSTM) network with filter mechanism (MALNet) to extract the text features and classify the relation of two entities in a sentence. In particular, we combine LSTM with attention mechanism to obtain the shallow local information and introduce a filter layer based on attention mechanism to strength the available information. Besides, we design a semantic rule for marking the key word between the target words and construct a key word layer to extract its semantic information. We evaluated the performance of our model on SemEval-2010 Task8 dataset and KBP-37 dataset. We achieved an F1-score of 86.3% on SemEval-2010 Task8 dataset and F1-score of 61.4% on KBP-37 dataset, which shows that our method is superior to the previous state-of-the-art methods.

show abstract

LSTM-CRF Neural Network With Gated Self Attention for Chinese NER

Cited by 49 publications

References 25 publications

MoGCN: Mixture of Gated Convolutional Neural Network for Named Entity Recognition of Chinese Historical Texts

MoGCN: Mixture of Gated Convolutional Neural Network for Named Entity Recognition of Chinese Historical Texts

POISE: Efficient Cross-Domain Chinese Named Entity Recognization via Transfer Learning

Attention-Based LSTM with Filter Mechanism for Entity Relation Classification

Contact Info

Product

Resources

About