Large-scale application of named entity recognition to biomedicine and epidemiology

Abstract

Background Despite significant advancements in biomedical named entity recognition methods, the clinical application of these systems continues to face many challenges: (1) most of the methods are trained on a limited set of clinical entities; (2) these methods are heavily reliant on a large amount of data for both pre-training and prediction, making their use in production impractical; (3) they do not consider non-clinical entities, which are also related to patient’s health, such as social, economic or demographic factors. Methods In this paper, we develop Bio-Epidemiology-NER (https://pypi.org/project/Bio-Epidemiology-NER/) an open-source Python package for detecting biomedical named entities from the text. This approach is based on a Transformer-based system and trained on a dataset that is annotated with many named entities (medical, clinical, biomedical, and epidemiological). This approach improves on previous efforts in three ways: (1) it recognizes many clinical entity types, such as medical risk factors, vital signs, drugs, and biological functions; (2) it is easily configurable, reusable, and can scale up for training and inference; (3) it also considers non-clinical factors (age and gender, race and social history and so) that influence health outcomes. At a high level, it consists of the phases: pre-processing, data parsing, named entity recognition, and named entity enhancement. Results Experimental results show that our pipeline outperforms other methods on three benchmark datasets with macro-and micro average F1 scores around 90 percent and above. Conclusion This package is made publicly available for researchers, doctors, clinicians, and anyone to extract biomedical named entities from unstructured biomedical texts.

RecSOI: recommending research directions using statements of ignorance

“…Because of that, we rely on concepts that can be extracted from the abstracts of the author using a named entity recognition tool. For the concepts heuristic, the concepts in the abstracts and in the SOIs are therefore retrieved using the named entity recognition tool from the work of Raza et al [ 31 ].…”

Improvement of Outpatient Service Processes: A Case Study of the University of Hong Kong-Shenzhen Hospital

“…Building textual codebook. Specifically, to establish a textual codebook, we use a pre-trained BioEN [35] to extract the related entities from the medical reports in the training set, where the related entities are divided into four entity groups, i.e., biological structure, detailed description, disease disorder, and sign symptom. Next, we compute the frequency of each entity and pick top κ 0 entities for each entity group as keywords in the textual codebook.…”

Constructing a disease database and using natural language processing to capture and standardize free text clinical information

“…Contributions Our proposed framework consists of a comprehensive pipeline that includes the creation of a high-quality database from published case reports, the design and implementation of NLP models to detect and examine clinical and non-clinical concepts in the data, and a thorough evaluation process. A named entity recognition (NER) algorithm 8 is included in the NLP models, and it is capable of accurately identifying essential clinical concepts such as diseases, conditions, symptoms, and drugs, as well as non-clinical concepts such as social determinants of health (SDOH) 9 . Furthermore, we developed a relation extraction (RE) model to identify relationships between these concepts, including disease-complication, treatment-improvement, and drug-adverse-effect associations.…”