Natural language processing diagnosed behavioral disturbance vs confusion assessment method for the intensive care unit: prevalence, patient characteristics, overlap, and association with treatment and outcome

Abstract

Purpose: To compare the prevalence, characteristics, drug treatment for delirium, and outcomes of patients with Natural Language Processing (NLP) diagnosed behavioral disturbance (NLP-Dx-BD) vs Confusion Assessment Method for intensive care unit (CAM-ICU) positivity. Methods:In three combined medical-surgical ICUs, we obtained data on demographics, treatment with antipsychotic medications, and outcomes. We applied NLP to caregiver progress notes to diagnose behavioral disturbance and analyzed simultaneous CAM-ICU. Results:We assessed 2313 patients with a median lowest Richmond Agitation-Sedation Scale (RASS) score of − 2 (− 4.0 to − 1.0) and median highest RASS score of 1 (0 to 1). Overall, 1246 (53.9%) patients were NLP-Dx-BD positive (NLP-Dx-BD pos ) and 578 (25%) were CAM-ICU positive (CAM-ICU pos ). Among NLP-Dx-BD pos patients, 539 (43.3%) were also CAM-ICU pos . In contrast, among CAM-ICU pos patients, 539 (93.3%) were also NLP-Dx-BD pos . The use of antipsychotic medications was highest in patients in the CAM-ICU pos and NLP-Dx-BD pos group (24.3%) followed by the CAM-ICU neg and NLP-Dx-BD pos group (10.5%). In NLP-Dx-BD neg patients, antipsychotic medication use was lower at 5.1% for CAM-ICU pos and NLP-Dx-BD neg patients and 2.3% for CAM-ICU neg and NLP-Dx-BD neg patients (overall P < 0.001). Regardless of CAM-ICU status, after adjustment and on time-dependent Cox modelling, NLP-Dx-BD was associated with greater antipsychotic medication use. Finally, regardless of CAM-ICU status, NLP-Dx-BD pos patients had longer duration of ICU and hospital stay and greater hospital mortality (all P < 0.001). Conclusion:More patients were NLP-Dx-BD positive than CAM-ICU positive. NLP-Dx-BD and CAM-ICU assessment describe partly overlapping populations. However, NLP-Dx-BD identifies more patients likely to receive antipsychotic medications. In the absence of NLP-Dx-BD, treatment with antipsychotic medications is rare.

Natural language processing diagnosed behavioural disturbance phenotypes in the intensive care unit: characteristics, prevalence, trajectory, treatment, and outcomes

“…We analysed these notes using NLP techniques. As previously described [ 8 , 15 ], each progress note was converted into sentence vectors, tokenised and searched for the presence of words indicative of agitated or non-agitated behaviour (Natural Language Toolkit; NLTK 3.5) [ 16 ], a process equivalent to the first step of Large Language Model Generative Pre-trained Transformer strategies recently popularised by OpenAI.…”

“…Such studies have also suggested that there might be clinical value in further investigating the characteristics, prevalence, trajectory, treatment, and outcomes of disturbed behavioural phenotypes, a similar but not identical concept to delirium [ 8 – 14 ]. In this regard, natural language processing (NLP) of caregivers’ notes has recently emerged as a screening tool for behaviour in critically ill patients [ 8 – 14 ]. NLP has also been recently used to describe the syndrome of NLP-diagnosed behaviour disturbance (NLP-Dx-BD) [ 15 ].…”

“…Different from delirium screening tools, the identification of this syndrome requires the application of NLP to screen caregiver notes for the purpose of detecting words or phrases that describe disturbed behaviour (see “ Methods ” section). Such NLP-Dx-BD is a condition observed in critically ill patients and has high sensitivity for the identification of patients who later go on to be treated with antipsychotic medications in the ICU [ 8 ].…”

Identifying encephalopathy in patients admitted to an intensive care unit: Going beyond structured information using natural language processing

“…In the field of ED, NLP has demonstrated that neither ICD codes or specific delirium clinical scales are sensitive enough to capture all the phenotypic range of encephalopathy. In particular, behavioural disturbances captured by NLP encloses more patients at risk of receiving antipsychotic medications or having higher morbimortality rates in the ICU than the group of patients defined by the CAM-ICU ( 17 ). Recent approaches have tried to incorporate keywords related to the delirium semiology into machine learning classifiers to label patients with delirium ( 10 , 18 ).…”

What language conveys distress and reassurance?

“…In this issue of Intensive Care Medicine [ 12 ], Young and colleagues present study results from an ICU cohort evaluated twice daily with the CAM-ICU. In parallel, behavioral disturbances were identified with natural language processing (NLP) using 24 previously validated words documented in ICU chart notes.…”