Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking

Murty, Shikhar; Verga, Patrick; Vilnis, Luke; Radovanovic, Irena; McCallum, Andrew

doi:10.18653/v1/p18-1010

Cited by 88 publications

(90 citation statements)

References 39 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We perform experiments on two benchmark Biocreative datasets as well as the recently introduced MedMentions data (Murty et al, 2018). Our experiments consider three types of models.…”

Section: Resultsmentioning

confidence: 99%

Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets

Greenberg¹,

Bansal²,

Verga³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

Extracting typed entity mentions from text is a fundamental component to language understanding and reasoning. While there exist substantial labeled text datasets for multiple subsets of biomedical entity types-such as genes and proteins, or chemicals and diseasesit is rare to find large labeled datasets containing labels for all desired entity types together. This paper presents a method for training a single CRF extractor from multiple datasets with disjoint or partially overlapping sets of entity types. Our approach employs marginal likelihood training to insist on labels that are present in the data, while filling in "missing labels". This allows us to leverage all the available data within a single model. In experimental results on the Biocreative V CDR (chemicals/diseases), Biocreative VI ChemProt (chemicals/proteins) and Med-Mentions (19 entity types) datasets, we show that joint training on multiple datasets improves NER F1 over training in isolation, and our methods achieve state-of-the-art results.

show abstract

“…We perform experiments on two benchmark Biocreative datasets as well as the recently introduced MedMentions data (Murty et al, 2018). Our experiments consider three types of models.…”

Section: Resultsmentioning

confidence: 99%

Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets

Greenberg¹,

Bansal²,

Verga³

et al. 2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…for various values of K nearest neighbours. Our candidate generator provides a 5% absolute improvement over Murty et al (2018) despite generating 46% fewer candidates per mention on average.…”

Section: Candidate Generationmentioning

confidence: 98%

“…This reduced subset is comprised of sections 0, 1, 2 and 9 (SNOMED) of the UMLS 2017 AA release, which are publicly distributable. It contains 2.78M unique concepts and covers 99% of the mention concepts present in the MedMentions dataset (Murty et al, 2018).…”

Section: Candidate Generation For Entity Linkingmentioning

confidence: 99%

ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

Neumann¹,

King²,

Beltagy³

et al. 2019

Proceedings of the 18th BioNLP Workshop and Shared Task

509

334

View full text Add to dashboard Cite

Despite recent advances in natural language processing, many statistical models for processing text perform extremely poorly under domain shift. Processing biomedical and clinical text is a critically important application area of natural language processing, for which there are few robust, practical, publicly available models. This paper describes scis-paCy, a new Python library and models for practical biomedical/scientific text processing, which heavily leverages the spaCy library. We detail the performance of two packages of models released in scispaCy and demonstrate their robustness on several tasks and datasets. Models and code are available at https:// allenai.github.io/scispacy/.

show abstract

“…Entity linking models Entity linking given mention boundaries as input can be broken into the tasks of candidate generation and candidate ranking. When frequency information or alias tables are unavailable, prior work has used measures of similarity of the mention string to entity names for candidate generation (Sil et al, 2012;Murty et al, 2018). For candidate ranking, recent work employed distributed representations of mentions in context and entity candidates and neural models to score their compatibility.…”

Section: Related Workmentioning

confidence: 99%

Zero-Shot Entity Linking by Reading Entity Descriptions

Logeswaran

Chang

Lee

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

217

288

View full text Add to dashboard Cite

We present the zero-shot entity linking task, where mentions must be linked to unseen entities without in-domain labeled data. The goal is to enable robust transfer to highly specialized domains, and so no metadata or alias tables are assumed. In this setting, entities are only identified by text descriptions, and models must rely strictly on language understanding to resolve the new entities. First, we show that strong reading comprehension models pre-trained on large unlabeled data can be used to generalize to unseen entities. Second, we propose a simple and effective adaptive pre-training strategy, which we term domainadaptive pre-training (DAP), to address the domain shift problem associated with linking unseen entities in a new domain. We present experiments on a new dataset that we construct for this task and show that DAP improves over strong pre-training baselines, including BERT. The data and code are available at https: //github.com/lajanugen/zeshel. 1

show abstract

Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking

Cited by 88 publications

References 39 publications

Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets

Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets

ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

Zero-Shot Entity Linking by Reading Entity Descriptions

Contact Info

Product

Resources

About