On Biomedical Named Entity Recognition: Experiments in Interlingual Transfer for Clinical and Social Media Texts

Abstract

Although deep neural networks yield state-of-the-art performance in biomedical named entity recognition (bioNER), much research shares one limitation: models are usually trained and evaluated on English texts from a single domain. In this work, we present a fine-grained evaluation intended to understand the efficiency of multilingual BERT-based models for bioNER of drug and disease mentions across two domains in two languages, namely clinical data and user-generated texts on drug therapy in English and Russian. We investigate the role of transfer learning (TL) strategies between four corpora to reduce the number of examples that have to be manually annotated. Evaluation results demonstrate that multi-BERT shows the best transfer capabilities in the zero-shot setting when training and test sets are either in the same language or in the same domain. TL reduces the amount of labeled data needed to achieve high performance on three out of four corpora: pretrained models reach 98-99% of the full dataset performance on both types of entities after training on 10-25% of sentences. We demonstrate that pretraining on data with one or both types of transfer can be effective.

Pre-trained Language Models in Biomedical Domain: A Systematic Survey

“…Instead of treating the BioNER as the sequence labeling problem, Sun et al [229] proposed to consider the BioNER as the machine reading comprehension (MRC) problem based on BERT. Besides English, there is much work exploring the pre-trained language models on the BioNER of other languages, including Chinese [43,91,129,130,252,272], Spanish [6,75,159], French [41], Korean [109], Russian [154], Arabic [23], Italian [28]. In Table 6, we summary the commonly used datasets in the BioNER task.…”

The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews

“…Following our previous work on NER (Miftahutdinov et al, 2020), we utilize different BERT models with a softmax layer over all possible tags as the output for NER. Word labels are encoded with the BIO tag scheme.…”

Comparative study of text representation and learning for Persian named entity recognition

“…NER is applied to various domains, and the set of entities is selected based on the given domain, for example, NER has been used to extract drug and disease mentions in English and Russian by Miftahutdinov and others [1]. In addition, Chen and others [2] investigated clinical NER tasks and compared different active learning algorithms for NER, and Carbonell and others [3] applied NER to handwritten data jointly with text localization and transcription.…”

“…These results represent state-of-the-art performance on the Persian NER task.Named entity recognition (NER) detects and classifies named entities in text into categories, for example, persons, organizations, locations, and time expressions. NER is a fundamental task in many natural language processing (NLP) applications, for example, information extraction, text summarization, and machine translation, especially for low-resource languages, for example, Persian, which requires additional features to process the text effectively.NER is applied to various domains, and the set of entities is selected based on the given domain, for example, NER has been used to extract drug and disease mentions in English and Russian by Miftahutdinov and others [1]. In addition, Chen and others [2] investigated…”