Dynamic Transfer Learning for Named Entity Recognition

Bhatia, Parminder; Arumae, Kristjan; Celikkaya, E. Busra

doi:10.1007/978-3-030-24409-5_7

Cited by 15 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, [33] showed that pre-training a BiLSTM-CRF model on a silver standard corpus of 5M sentences from PubMed abstracts, tagged using a trained CRF model rather than human experts, boosts performance on downstream biomedical NER tasks for different entity types. Other work, including [34][35][36][37][38], explore other variations of transfer learning and come to similar conclusions that transfer learning can significantly improve performance on downstream NER tasks. We extend these previous works by (1) comparing the effectiveness of three NER pre-training corpora of differing size and quality and (2) incorporating semi-supervised learning after transfer learning to further improve the performance of our NER approaches.…”

Section: Related Workmentioning

confidence: 77%

A pre-training and self-training approach for biomedical named entity recognition

et al. 2021

View full text Add to dashboard Cite

Named entity recognition (NER) is a key component of many scientific literature mining tasks, such as information retrieval, information extraction, and question answering; however, many modern approaches require large amounts of labeled training data in order to be effective. This severely limits the effectiveness of NER models in applications where expert annotations are difficult and expensive to obtain. In this work, we explore the effectiveness of transfer learning and semi-supervised self-training to improve the performance of NER models in biomedical settings with very limited labeled data (250-2000 labeled samples). We first pre-train a BiLSTM-CRF and a BERT model on a very large general biomedical NER corpus such as MedMentions or Semantic Medline, and then we fine-tune the model on a more specific target NER task that has very limited training data; finally, we apply semi-supervised self-training using unlabeled data to further boost model performance. We show that in NER tasks that focus on common biomedical entity types such as those in the Unified Medical Language System (UMLS), combining transfer learning with self-training enables a NER model such as a BiLSTM-CRF or BERT to obtain similar performance with the same model trained on 3x-8x the amount of labeled data. We further show that our approach can also boost performance in a low-resource application where entities types are more rare and not specifically covered in UMLS.

show abstract

Section: Related Workmentioning

confidence: 77%

A pre-training and self-training approach for biomedical named entity recognition

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Note that, when an entity mention is a single word, the function E will behave exactly like the original mapping C. In this paper, we investigate two different approaches for obtaining embedded representations of named entities. The first one, named BERT s , consider a named entity as a document 3 . In the second one, named BERT t , we further investigate the model by extracting the embeddings of single words within the context of the sentence.…”

Section: The Proposed Solutionmentioning

confidence: 99%

“…The Bi-LSTM network is coupled with the use of CRF models for sequence labeling in source and target domains separately to avoid annotation efforts. Additionally, Bhatia et al presented a framework in [3] for performing named entity recognition for domains with low resources such as medicinal texts. They proposed a tunable transfer learning architecture to counter the data scarcity problem, coupled with a parameter sharing approach to transfer overlapped representation from the source to the target domain.…”

Section: Related Workmentioning

confidence: 99%

LearningToAdapt with word embeddings: Domain adaptation of Named Entity Recognition systems

Nozza

Manchanda

Fersini

et al. 2021

Information Processing & Management

View full text Add to dashboard Cite

The task of Named Entity Recognition (NER) is aimed at identifying named entities in a given text and classifying them into pre-defined domain entity types such as persons, organizations, locations. Most of the existing NER systems make use of generic entity type classification schemas, however, the comparison and integration of (more or less) different entity types among different NER systems is a complex problem even for human experts. In this paper, we propose a supervised approach called L2AWE (Learning To Adapt with Word Embeddings) which aims at adapting a NER system trained on a source classification schema to a given target one. In particular, we validate the hypothesis that the embedding representation of named entities can improve the semantic meaning of the feature space used to perform the adaptation from a source to a target domain. The results obtained on benchmark datasets of informal text show that L2AWE not only outperforms several state of the art models, but it is also able to tackle errors and uncertainties given by NER systems.

show abstract

“…Transfer learning has been implemented in various different machine learning tasks, achieving notable results, for instance, textual summarization [4], named entity recognition [5], question answering [6,7], and text classification [8].…”

Section: Transfer Learning In Nlpmentioning

confidence: 99%

Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

Sarhan

Spruit

2020

Applied Sciences

View full text Add to dashboard Cite

Various tasks in natural language processing (NLP) suffer from lack of labelled training data, which deep neural networks are hungry for. In this paper, we relied upon features learned to generate relation triples from the open information extraction (OIE) task. First, we studied how transferable these features are from one OIE domain to another, such as from a news domain to a bio-medical domain. Second, we analyzed their transferability to a semantically related NLP task, namely, relation extraction (RE). We thereby contribute to answering the question: can OIE help us achieve adequate NLP performance without labelled data? Our results showed comparable performance when using inductive transfer learning in both experiments by relying on a very small amount of the target data, wherein promising results were achieved. When transferring to the OIE bio-medical domain, we achieved an F-measure of 78.0%, only 1% lower when compared to traditional learning. Additionally, transferring to RE using an inductive approach scored an F-measure of 67.2%, which was 3.8% lower than training and testing on the same task. Hereby, our analysis shows that OIE can act as a reliable source task.

show abstract

Dynamic Transfer Learning for Named Entity Recognition

Cited by 15 publications

References 25 publications

A pre-training and self-training approach for biomedical named entity recognition

A pre-training and self-training approach for biomedical named entity recognition

LearningToAdapt with word embeddings: Domain adaptation of Named Entity Recognition systems

Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

Contact Info

Product

Resources

About