Enhanced neurologic concept recognition using a named entity recognition model based on transformers

Abstract

Although deep learning has been applied to the recognition of diseases and drugs in electronic health records and the biomedical literature, relatively little study has been devoted to the utility of deep learning for the recognition of signs and symptoms. The recognition of signs and symptoms is critical to the success of deep phenotyping and precision medicine. We have developed a named entity recognition model that uses deep learning to identify text spans containing neurological signs and symptoms and then maps these text spans to the clinical concepts of a neuro-ontology. We compared a model based on convolutional neural networks to one based on bidirectional encoder representation from transformers. Models were evaluated for accuracy of text span identification on three text corpora: physician notes from an electronic health record, case histories from neurologic textbooks, and clinical synopses from an online database of genetic diseases. Both models performed best on the professionally-written clinical synopses and worst on the physician-written clinical notes. Both models performed better when signs and symptoms were represented as shorter text spans. Consistent with prior studies that examined the recognition of diseases and drugs, the model based on bidirectional encoder representations from transformers outperformed the model based on convolutional neural networks for recognizing signs and symptoms. Recall for signs and symptoms ranged from 59.5% to 82.0% and precision ranged from 61.7% to 80.4%. With further advances in NLP, fully automated recognition of signs and symptoms in electronic health records and the medical literature should be feasible.

Inter-rater agreement for the annotation of neurologic signs and symptoms in electronic health records

“…Our motivation for tagging signs and symptoms by the length and type of the text span was a hypothesis that neural networks trained to recognize signs and symptoms in medical text would exhibit lower accuracies with longer text spans. This hypothesis was confirmed by a recent study from our group ( 18 ).…”

“…Furthermore, other neural networks are likely to outperform the convolutional neural network (CNN), which is the baseline for Prodigy. We have found that a neural network based on bidirectional encoder representations from transformers (BERT) can improve performance on the text span task by 5 to 10% ( 18 ). Others have found that deep learning approaches based on BERT outperform approaches based on CNN for concept identification and extraction tasks ( 17 ).…”

“…NN was trained on 11,000 manually annotated sentences derived from neurology textbooks, online neurological disease descriptions, and electronic health record notes. Further details on training the NN are available in ( 18 ).…”

“…Arbabi et al developed a convolutional neural network that matches input phrases to concepts in the Human Phenotype Ontology with high accuracy ( 16 ). Other deep learning approaches, including neural networks based on bidirectional encoder representations from transformers (BERT), show promise for automated clinical concept extraction ( 5 , 6 , 17 , 18 ).…”

Improving Neurology Clinical Care With Natural Language Processing Tools

“…18 In addition, NLP can support the creation of neurology-specific ontologies and standardized vocabularies, facilitating better information exchange and interoperability across different health care systems and research institutions. 19 Applications of NLP models can also be expanded to inpatient and outpatient operations and be suitable in the creation of new data streams (eAppendix 1, links.lww.com/WNL/D179). The below discussion entails incorporations of NLP tools in clinical practice, which if entertained in future neurology health care systems would most certainly require validation through rigorous prospective and randomized studies.…”