Utilizing BERT for biomedical and clinical text mining

Zhu, Runjie; Tu, Xinhui; Huang, Jimmy Xiangji

doi:10.1016/b978-0-12-819314-3.00005-7

Cited by 16 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Named entity recognition models based on deep learning can recognize neurologic signs and symptoms in the biomedical literature and electronic health records (Table 1). Previous work has shown that BERT outperforms CNNs on recognizing drugs and diseases in annotated test corpora (52,55). We Text spans that identified clinical concepts were longer in the EHR corpus and shortest in the OMIM corpus (see blue inset histograms).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Enhanced neurologic concept recognition using a named entity recognition model based on transformers

Azizi

Hier

Wunsch

2022

Front. Digit. Health

View full text Add to dashboard Cite

Although deep learning has been applied to the recognition of diseases and drugs in electronic health records and the biomedical literature, relatively little study has been devoted to the utility of deep learning for the recognition of signs and symptoms. The recognition of signs and symptoms is critical to the success of deep phenotyping and precision medicine. We have developed a named entity recognition model that uses deep learning to identify text spans containing neurological signs and symptoms and then maps these text spans to the clinical concepts of a neuro-ontology. We compared a model based on convolutional neural networks to one based on bidirectional encoder representation from transformers. Models were evaluated for accuracy of text span identification on three text corpora: physician notes from an electronic health record, case histories from neurologic textbooks, and clinical synopses from an online database of genetic diseases. Both models performed best on the professionally-written clinical synopses and worst on the physician-written clinical notes. Both models performed better when signs and symptoms were represented as shorter text spans. Consistent with prior studies that examined the recognition of diseases and drugs, the model based on bidirectional encoder representations from transformers outperformed the model based on convolutional neural networks for recognizing signs and symptoms. Recall for signs and symptoms ranged from 59.5% to 82.0% and precision ranged from 61.7% to 80.4%. With further advances in NLP, fully automated recognition of signs and symptoms in electronic health records and the medical literature should be feasible.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Although considerable work has been done on automated concept identification of drugs and diseases, less work has been done on the automated identification of signs and symptoms (52). Identifying signs and symptoms is critical to precision medicine and deep phenotyping (56).…”

Section: A Proposed Approachmentioning

confidence: 99%

Enhanced neurologic concept recognition using a named entity recognition model based on transformers

Azizi

Hier

Wunsch

2022

Front. Digit. Health

View full text Add to dashboard Cite

show abstract

“…14 This was a significant leap forward in state-of-the-art deep learning-based natural language processing as it allowed for the development of large foundational models (eg, Bidirectional Encoder Representations from Transformers [BERT] 15 and Generative Pretrained Transformer 3 [GPT-3]), 16 pretrained on massive amounts of text data from the Internet. There are now several publicly available Transformer-based language models adapted to the clinical domain through further training on biomedical corpora (eg, ClinicalBERT, 17 , 18 GatorTtron, 19 Clincal-Longformer 20 ). A frequently cited obstacle to creating AI systems is access to labeled data.…”

Section: Proposed Alternativementioning

confidence: 99%

Could an artificial intelligence approach to prior authorization be more human?

Lenert

Lane

Wehbe

2023

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

Prior authorization (PA) may be a necessary evil within the healthcare system, contributing to physician burnout and delaying necessary care, but also allowing payers to prevent wasting resources on redundant, expensive, and/or ineffective care. PA has become an “informatics issue” with the rise of automated methods for PA review, championed in the Health Level 7 International’s (HL7’s) DaVinci Project. DaVinci proposes using rule-based methods to automate PA, a time-tested strategy with known limitations. This article proposes an alternative that may be more human-centric, using artificial intelligence (AI) methods for the computation of authorization decisions. We believe that by combining modern approaches for accessing and exchanging existing electronic health data with AI methods tailored to reflect the judgments of expert panels that include patient representatives, and refined with “few shot” learning approaches to prevent bias, we could create a just and efficient process that serves the interests of society as a whole. Efficient simulation of human appropriateness assessments from existing data using AI methods could eliminate burdens and bottlenecks while preserving PA’s benefits as a tool to limit inappropriate care.

show abstract

“…Data used by NLP models is pre-processed using techniques such as lemmatisation and stemming – reducing words to a single root for uniformity, keyword extraction – extracting important phrases and keywords, named entity recognition – classification of named entities into predefined categories, and tokenisation – breaking down text into smaller chunks. NLP algorithms perform language processing tasks such as natural language inference, entity extraction, relation extraction, text classification, question answering and text summarization [12] . Such tasks are explained further in Section 3 , and are used as performance indicators of the models compared and discussed in this review.…”

Section: Introductionmentioning

confidence: 99%