Ascertainment of Delirium Status Using Natural Language Processing From Electronic Health Records

Abstract

BACKGROUND Delirium is underdiagnosed in clinical practice, and is not routinely coded for billing. Manual chart review can be used to identify the occurrence of delirium, however, it is labor-intensive and impractical for large-scale studies. Natural language processing (NLP) has the capability to process raw text in electronic health records (EHRs) and determine the meaning of the information. We developed and validated NLP algorithms to automatically identify the occurrence of delirium from EHRs. METHODS This study used a randomly selected cohort from the population-based Mayo Clinic Biobank (n=300, age>=65). We adopted the standardized evidence-based framework confusion assessment method (CAM) to develop and evaluate NLP algorithms to identify the occurrence of delirium using clinical notes in EHRs. Two NLP algorithms were developed based on CAM criteria; one based on the original CAM (NLP-CAM; delirium vs. no delirium) and another based on our modified CAM (NLP-mCAM; definite, possible, and no delirium). The sensitivity, specificity, and accuracy were used for concordance in delirium status between NLP algorithms and manual chart review as the gold standard. The prevalence of delirium cases was examined using ICD-9, NLP-CAM, and NLP-mCAM. RESULTS NLP-CAM demonstrated a sensitivity, specificity and accuracy of 0.919, 1.000 and 0.967, respectively. NLP-mCAM demonstrated sensitivity, specificity and accuracy of 0.827, 0.913 and 0.827, respectively. The prevalence analysis of delirium showed that the NLP-CAM algorithm identified 12,651 (9.4%) delirium patients, the NLP-mCAM algorithm identified 20,611 (15.3%) definite delirium cases and 10,762 (8.0%) possible cases.

The Implication of Latent Information Quality to the Reproducibility of Secondary Use of Electronic Health Records

“…Due to the variability of diagnostic methods (i.e., no singular, conclusive diagnostic test) in the clinical setting, the documentation patterns of delirium-related findings can be variable. As suggested by prior studies, the following definition was applied to determine patients’ delirium status ( Table 2 ): the presence of International Classification of Diseases (ICD) codes for delirium, the presence of a nursing flowsheet documentation of their assessment of delirium, and the presence of CAM definition based on information extracted from clinical notes (CAM-NLP) [ 17 ].…”

Delirium occurrence and association with outcomes in hospitalized COVID-19 patients

“…Delirium was diagnosed if one of the three criteria were met: 1) Physician diagnosis of delirium or encephalopathy by International classification of disease (ICD)-9/10 coding, or 2) Nursing flowsheet documentation of Confusion Assessment Method (CAM) tool (performed twice daily on all hospitalized patients), or 3) Using validated natural language processing (NLP) system based on CAM criteria (Fu, 2020). A history of delirium was established using the same eligibility of meeting one of the three above criteria for delirium prior to the COVID-19 diagnosis date or hospital admission.…”

Identifying Patients With Delirium Based on Unstructured Clinical Notes: Observational Study

“…One such study [ 20 ] summarized patterns in the delirium literature over time, using unsupervised learning methods; by contrast, our work used NLP to extract information from clinical notes. Another study [ 21 ] detected delirium using an open-source NLP pipeline MedTaggerIE—an unstructured information management architecture–based information extraction framework. Shao et al [ 22 ] experimented with 3 different topic modeling methods and a keyword search method for identifying delirium-related documents and sentences in clinical notes.…”