Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning

Peng, Minlong; Xing, Xiaoyu; Zhang, Qi; Fu, Jinlan; Huang, Xuanjing

doi:10.18653/v1/p19-1231

Cited by 87 publications

(84 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are also a lot of weak labels lying on the web or gazetteers, which have not been explored. Consequently, a number of works focus on distantly supervised methods, using anchors or gazetteers to generate data by distant supervision (Liu et al, 2015;Cao et al, 2019;Peng et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

“…Then we step into the second phase (i.e., NEE) in which the model is trained to extract typed entities with gazetteer-labeled data. Peng et al, 2019). A standard strategy is to scan through the anchor text in D g using the gazetteer of a given entity type y and treat anchors matched with entries of the given gazetteer as the entities with type y.…”

Section: Named Entity Extractionmentioning

confidence: 99%

“…One can collect them from online resources, such as the Wikipedia anchors and gazetteers (named entity dictionaries). Although automatically derived corpora usually contain massive noisy data, it still contains some extend the valuable semantic information required for NER (Peng et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Coarse-to-Fine Pre-training for Named Entity Recognition

Mei¹,

Yu²,

Zhang³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

More recently, Named Entity Recognition has achieved great advances aided by pre-training approaches such as BERT. However, current pre-training techniques focus on building language modeling objectives to learn a general representation, ignoring the named entityrelated knowledge. To this end, we propose a NER-specific pre-training framework to inject coarse-to-fine automatically mined entity knowledge into pre-trained models. Specifically, we first warm-up the model via an entity span identification task by training it with Wikipedia anchors, which can be deemed as general-typed entities. Then we leverage the gazetteer-based distant supervision strategy to train the model extract coarse-grained typed entities. Finally, we devise a self-supervised auxiliary task to mine the fine-grained named entity knowledge via clustering. Empirical studies on three public NER datasets demonstrate that our framework achieves significant improvements against several pre-trained baselines, establishing the new state-of-the-art performance on three benchmarks. Besides, we show that our framework gains promising results without using human-labeled training data, demonstrating its effectiveness in labelfew and low-resource scenarios. 1

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Named Entity Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Coarse-to-Fine Pre-training for Named Entity Recognition

Mei¹,

Yu²,

Zhang³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…Distant-LSTM-CRF [3] propose for the distantly supervised aspect term extraction, which can be viewed as an entity recognition task of a single type for business reviews. AdaPU [16] propose algorithm using unlabeled data and dictionary to perform NER tasks while using AdaSampling way to expand the named entity recognition dictionary.…”

Section: Related Workmentioning

confidence: 99%

“…The method then uses the classifier to perform unsupervised aspect term extraction by training on the auto-tagged datasets obtained by the method. AdaPU [16] explored ways to perform NER using only unlabeled data and a dictionary of named entities. The method represents the task as a Positive Unlabeled (PU) learning problem, and proposes a PU learning algorithm to perform the task.…”

Section: Distant-lstm-crf [3]mentioning

confidence: 99%

Improving Distantly-Supervised Named Entity Recognition for Traditional Chinese Medicine Text via a Novel Back-Labeling Approach

Zhang

Xia

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Recent advances in deep neural networks (DNNs) have enabled us to achieve reliable named entity recognition (NER) models without handcrafting features. However, these are also some obstacles imposed by using those machine learning methods, in need of a large amount of manually labeled data. To avoid such limitations, we could replace human annotation with distant supervision, however there remain a technical challenge on the error label issue caused by ignoring the entities that are not included in the vocabulary, which should be addressed to achieve the effective NER model. Then, we propose a novel backlabeling approach and integrate it into a tagging scheme, especially, we apply this scheme to handle the NER task in traditional Chinese medicine (TCM) field. In addition, we discuss how to use distant supervision methods to achieve better performance of the NER model. We conduct some experiments and verify that our scheme can effectively improve the entity recognition on the basis of distant supervision.

show abstract

An MRC and adaptive positive‐unlabeled learning framework for incompletely labeled named entity recognition

Wang

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

Currently, named entity recognition (NER) is mainly evaluated on standard and well-annotated data sets.However, the construction of a well-annotated data set will consume a lot of manpower and time. In lots of applications of NER, data sets may contain a lot of noise, and a large part of noise comes from unlabeled entities.At present, the training process of most models treat unlabeled entities as nonentities, which causes these models to lean toward predicting most words of an input context as nonentities and greatly affects their performances. In this paper, as the first attempt, we innovatively propose an adaptive positive-unlabeled (adaPU) learning technology, and integrate the adaPU into a machine reading comprehension (MRC) framework for NER, which can still perform well on data sets with a large proportion of unlabeled entities. In our framework, to leverage the above problem that a model may predict most words of an input context as nonentities, we propose an adaPU learning technology by adjusting a loss coefficient of positive and negative samples. Moreover, instead of just constructing a fixed query for each entity type as input to MRC, we propose a new method of dynamically constructing multiple queries for each entity type, which also brings slight performance improvement for NER. Accordingly, we explore new training and entity inference strategies for

show abstract

Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning

Cited by 87 publications

References 35 publications

Coarse-to-Fine Pre-training for Named Entity Recognition

Coarse-to-Fine Pre-training for Named Entity Recognition

Improving Distantly-Supervised Named Entity Recognition for Traditional Chinese Medicine Text via a Novel Back-Labeling Approach

An MRC and adaptive positive‐unlabeled learning framework for incompletely labeled named entity recognition

Contact Info

Product

Resources

About