Prediction of emergency department resource requirements during triage: An application of current natural language processing techniques

Sterling, Nicholas W.; Brann, Felix; Patzer, Rachel E.; Di, Mengyu; Koebbe, Megan; Burke, Madalyn; Schrager, Justin D.

doi:10.1002/emp2.12253

Cited by 12 publications

(33 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Three studies compared the accuracy of triage scores assigned by multimodal models incorporating NLP to triage scores assigned by nurses. 17,36,47 Such models were more accurate than nurses in two out of three papers. 36,47 The addition of text data compared to structured data alone improved performance in assigning triage score.…”

Section: Prediction Of Triage Scorementioning

confidence: 90%

“…This free-text may be more expressive, nuanced, and contain a higher level of information than structured data. 17 Prior work has suggested that incorporating free-text may improve the performance of ML at ED triage and is an important area for future research despite the challenges of incorporating free-text data into models. [18][19][20] Natural language processing (NLP) uses computational methods to analyse and understand human language and its structure.…”

Section: (Which Was Not Certified By Peer Review) Preprintmentioning

confidence: 99%

“…There were 18 retrospective studies. 17,18,30,[34][35][36][37][38][39][40][41][42][43][44][45][46][47][48] One study reported their ML model was developed using retrospective data then validated using prospective data. 49 All used observational cohort designs.…”

Section: Characteristics Of Included Studiesmentioning

confidence: 99%

“…CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) Sterling et al 2020 noted the difficulty in capturing the general clinical impression of the triage nurse. 17 Ivanov et al also noted that important contextual aspects at triage were not available for consideration by ML models.…”

Section: Incorporating Clinical Gestaltmentioning

confidence: 99%

“…(which was not certified by peer review) Sterling et al 2020 noted the difficulty in capturing the general clinical impression of the triage nurse. 17 Ivanov et al also noted that important contextual aspects at triage were not available for consideration by ML models. 47 Future work could assess the impact of incorporating triage nurses' gestalt into predictive models.…”

Section: Incorporating Clinical Gestaltmentioning

confidence: 99%

See 4 more Smart Citations

Applications of Natural Language Processing at Emergency Department Triage: A Systematic Review

Stewart

Lü

Goudie

et al. 2022

Preprint

View full text Add to dashboard Cite

INTRODUCTION Millions of patients attend emergency departments (EDs) around the world every year. Patients are triaged on arrival by a trained nurse who collects structured data and an unstructured free-text history of presenting complaint. Natural language processing (NLP) uses various computational methods to analyse and understand human language, and has been applied to data acquired at ED triage to predict various outcomes. The objective of this systematic review is to evaluate how NLP has been applied to ED triage, assess if NLP based models outperform humans or current risk stratification techniques, and assess if incorporating free-text improve predictive performance of models when compared to predictive models that use only structured data. METHODS All English language peer-reviewed research that applied an NLP technique to free-text obtained at ED triage was eligible for inclusion. We excluded studies focusing solely on disease surveillance, and studies that used information obtained after triage. We searched the electronic databases MEDLINE, Embase, Cochrane Database of Systematic Reviews, Web of Science, and Scopus for medical subject headings and text keywords related to NLP and triage. Databases were last searched on 01/01/2022. Risk of bias in studies was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). Due to the high level of heterogeneity between studies, a metanalysis was not conducted. Instead, a narrative synthesis is provided. RESULTS In total, 3584 studies were screened, and 19 studies were included. The population size varied greatly between studies ranging from 1.8 million patients to 762 simulated encounters. The most common primary outcomes assessed were prediction of triage score, prediction of admission, and prediction of critical illness. NLP models achieved high accuracy in predicting need for admission, critical illness, and mapping free-text chief complaints to structured fields. Overall, NLP models predicted admission with greater accuracy than emergency physicians, outperformed abnormal vital sign trigger and triage score at predicting critical illness, and were more accurate than nurses at assigning triage scores in two out of three papers. Incorporating both structured data and free-text data improved results when compared to models that used only structured data. The majority of studies were (79%) were assessed to have a high risk of bias, and only one study reported the deployment of an NLP model into clinical practice. CONCLUSION Unstructured free-text triage notes contain valuable information that can be used by NLP models to predict clinically relevant outcomes. The use of NLP at ED triage appears feasible and could allow for early and accurate prediction of multiple important patient-oriented outcomes. However, there are few examples of implementation of into clinical practice, most research in retrospective, and the potential benefits of NLP at triage are yet to be realised.

show abstract

Section: Prediction Of Triage Scorementioning

confidence: 90%

Section: (Which Was Not Certified By Peer Review) Preprintmentioning

confidence: 99%

Section: Characteristics Of Included Studiesmentioning

confidence: 99%

Section: Incorporating Clinical Gestaltmentioning

confidence: 99%

Section: Incorporating Clinical Gestaltmentioning

confidence: 99%

See 3 more Smart Citations

Applications of Natural Language Processing at Emergency Department Triage: A Systematic Review

Stewart

Lü

Goudie

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

From crisis to opportunity: advancements in emergency language services

Guo,

Xiao,

Guo

2024

Humanit Soc Sci Commun

View full text Add to dashboard Cite

Natural Language Processing of Nursing Notes

Mitha

Schwartz

Hobensack

et al. 2022

CIN: Computers, Informatics, Nursing

View full text Add to dashboard Cite

Natural language processing includes a variety of techniques that help to extract meaning from narrative data. In healthcare, medical natural language processing has been a growing field of study; however, little is known about its use in nursing. We searched PubMed, EMBASE, and CINAHL and found 689 studies, narrowed to 43 eligible studies using natural language processing in nursing notes. Data related to the study purpose, patient population, methodology, performance evaluation metrics, and quality indicators were extracted for each study. The majority (86%) of the studies were conducted from 2015 to 2021. Most of the studies (58%) used inpatient data. One of four studies used data from open-source databases. The most common standard terminologies used were the Unified Medical Language System and Systematized Nomenclature of Medicine, whereas nursing-specific standard terminologies were used only in eight studies. Full system performance metrics (eg, F score) were reported for 61% of applicable studies. The overall number of nursing natural language processing publications remains relatively small compared with the other medical literature. Future studies should evaluate and report appropriate performance metrics and use existing standard nursing terminologies to enable future scalability of the methods and findings.

show abstract

Prediction of emergency department resource requirements during triage: An application of current natural language processing techniques

Cited by 12 publications

References 13 publications

Applications of Natural Language Processing at Emergency Department Triage: A Systematic Review

Applications of Natural Language Processing at Emergency Department Triage: A Systematic Review

From crisis to opportunity: advancements in emergency language services

Natural Language Processing of Nursing Notes

Contact Info

Product

Resources

About