Biomedical Named Entity Recognition at Scale

Kocaman, Veysel; Talby, David

doi:10.1007/978-3-030-68763-2_48

Cited by 43 publications

(23 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accordingly to [9], precision was computed as the percent- State-of-the-art (SOTA) performance. For NCBI-disease, SOTA1 is BioBERT, [5], SOTA2 is Spark NLP, [18], SOTA3 is BioFLAIR, [19]. For BC5CDR, SOTA1 is RL+DS+PA, [20], SOTA2 is Spark NLP and SOTA3 is BioFLAIR.…”

Section: Resultsmentioning

confidence: 99%

ELECTRAMed: a new pre-trained language representation model for biomedical NLP

Miolo,

Mantoan,

Orsenigo

2021

Preprint

View full text Add to dashboard Cite

The overwhelming amount of biomedical scientific texts calls for the development of effective language models able to tackle a wide range of biomedical natural language processing (NLP) tasks. The most recent dominant approaches are domain-specific models, initialized with general-domain textual data and then trained on a variety of scientific corpora. However, it has been observed that for specialized domains in which large corpora exist, training a model from scratch with just in-domain knowledge may yield better results. Moreover, the increasing focus on the compute costs for pre-training recently led to the design of more efficient architectures, such as ELECTRA. In this paper, we propose a pre-trained domain-specific language model, called ELECTRAMed, suited for the biomedical field. The novel approach inherits the learning framework of the general-domain ELECTRA architecture, as well as its computational advantages. Experiments performed on benchmark datasets for several biomedical NLP tasks support the usefulness of ELECTRAMed, which sets the novel state-of-the-art result on the BC5CDR corpus for named entity recognition, and provides the best outcome in 2 over the 5 runs of the 7th BioASQ-factoid Challange for the question answering task.

show abstract

Section: Resultsmentioning

confidence: 99%

ELECTRAMed: a new pre-trained language representation model for biomedical NLP

Miolo,

Mantoan,

Orsenigo

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In our previous study [16], we showed through extensive experiments that NER module in Spark NLP library exceeds the biomedical NER benchmarks reported by Stanza in 7 out of 8 benchmark datasets and in every dataset reported by SciSpacy without using heavy contextual embeddings like BERT. Using the modified version of the well known BiLSTM-CNN-Char NER architecture [17] into Spark environment, we also presented that even with a general purpose GloVe embeddings (GloVe6B) and with no lexical features, we were able to achieve state-of-the-art results in biomedical domain and produces better results than Stanza in 4 out of 8 benchmark datasets (Table Table 1).…”

Section: The Impact To Research Fieldsmentioning

confidence: 93%

Spark NLP: Natural Language Understanding at Scale

Kocaman

Talby

2021

Software Impacts

Self Cite

View full text Add to dashboard Cite

Spark NLP is a Natural Language Processing (NLP) library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines that can scale easily in a distributed environment. Spark NLP comes with 1100+ pretrained pipelines and models in more than 192+ languages. It supports nearly all the NLP tasks and modules that can be used seamlessly in a cluster. Downloaded more than 2.7 million times and experiencing 9x growth since January 2020, Spark NLP is used by 54% of healthcare organizations as the world's most widely used NLP library in the enterprise.

show abstract

“…CRF has the advantage over other ML algorithms to efficiently model dependencies between observations and labels, taking context into account. Among traditional ML algorithms, CRF is known as the most popular solution for solving ST tasks such as BioNER [9].…”

Section: Conditional Random Fieldmentioning

confidence: 99%

A Comparison between Named Entity Recognition Models in the Biomedical Domain

Cariello¹,

Lenci²,

Mitkov³

2021

Proceedings of the Translation and Interpreting Technology Online Conference TRITON 2021

View full text Add to dashboard Cite

The domain-specialised application of Named Entity Recognition (NER) is known as Biomedical NER (BioNER), which aims to identify and classify biomedical concepts that are of interest to researchers, such as genes, proteins, chemical compounds, drugs, mutations, diseases, and so on. The BioNER task is very similar to general NER but recognising Biomedical Named Entities (BNEs) is more challenging than recognising proper names from newspapers due to the characteristics of biomedical nomenclature. In order to address the challenges posed by BioNER, seven machine learning models were implemented comparing a transfer learning approach based on fine-tuned BERT with Bi-LSTM based neural models and a CRF model used as baseline. Precision, Recall and F1-score were used as performance scores evaluating the models on two well-known biomedical corpora: JNLPBA and BIOCREATIVE IV (BC-IV). Strict and partial matching were considered as evaluation criteria. The reported results show that a transfer learning approach based on fine-tuned BERT outperforms all others methods achieving the highest scores for all metrics on both corpora.

show abstract

Biomedical Named Entity Recognition at Scale

Cited by 43 publications

References 30 publications

ELECTRAMed: a new pre-trained language representation model for biomedical NLP

ELECTRAMed: a new pre-trained language representation model for biomedical NLP

Spark NLP: Natural Language Understanding at Scale

A Comparison between Named Entity Recognition Models in the Biomedical Domain

Contact Info

Product

Resources

About