Machine learning‐assisted screening for cognitive impairment in the emergency department

Yadgir, Simon R.; Engstrom, Collin; Jacobsohn, Gwen Costa; Green, Rebecca K.; Jones, Courtney Marie Cora; Cushman, Jeremy T.; Caprio, Thomas V.; Kind, Amy J H; Lohmeier, Michael; Shah, Manish N.; Patterson, Brian W.

doi:10.1111/jgs.17491

Cited by 12 publications

(15 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our model generated a very high NPV at the 90 th percentile for both BA Veterans (0.98 [0.96, 0.99]) and WA Veterans (0.98 [0.94, 0.99]). These findings are similar to the NPVs reported with the eRADAR tool in an EHR sample that was 89% WA (Barnes et al, 2020) but higher than the NPV reported by Yadgir et al (i.e., 0.93) (Yadgir et al, 2022). The PPV in our study was low for both BA Veterans (0.26 [0.21,0.31]) and WA Veterans (0.15 [0.12,0.20]) at the 90 th percentile cutoff.…”

Section: Discussionsupporting

confidence: 87%

“…Researchers in the UK developed models (including SVM) to identify patients with dementia (Jammeh et al, 2018), and Kaiser Permanente/UCSF researchers developed the eRADAR tool in research participants and then validated it in two health-care systems (Barnes et al, 2020, Coley et al, 2022); both studies limited their EHR interrogations to structured data and have shown some success in identifying undiagnosed dementia. Likewise, Yadgir et al used ML to identify structured variables associated with cognitive impairment in ER patients (Yadgir et al, 2022). Conversely, Boustani et al have developed passive digital signatures for ADRD by searching for predetermined variables in both structured and unstructured EHR data, and their work suggests that the combination can improve AUC by up to .11 (Boustani et al, 2020); however, like Barnes et al, Boustani et al use curated, preselected search terms rather than leveraging the potential of supervised ML to identify topic features associated with dementia.…”

Section: Discussionmentioning

confidence: 99%

“…Practically, this means that at that threshold, about a quarter of the BAs and a seventh of the WAs who were flagged by our model as having potential dementia would actually have dementia according to our manual chart reviews. In contrast, Yadgir et al, achieved PPVs greater than 0.4, but to do so, they applied threshold cutoffs higher than 0.8; this meant that they obtained a high true positive rate at the expense of low sensitivity, which is not optimal as a screening instrument given the high cutoff scores (Yadgir et al, 2022). Our algorithms compare very favorably to the eRADAR tool for dementia, which had a PPV of 0.115 in a research setting and 0.020 to 0.048 in patients (Barnes et al, 2020, Coley et al, 2022).…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records

Shao

Todd

Millard

et al. 2023

Preprint

View full text Add to dashboard Cite

The application of machine learning (ML) tools in electronic health records (EHRs) can help reduce the underdiagnosis of dementia, but models that are not designed to reflect minority population may perpetuate that underdiagnosis. To address the underdiagnosis of dementia in both Black Americans (BAs) and white Americans (WAs), we sought to develop and validate ML models that assign race-specific risk scores. These scores were used to identify undiagnosed dementia in BA and WA Veterans in EHRs. More specifically, risk scores were generated separately for BAs (n=10K) and WAs (n=10K) in training samples of cases and controls by performing ML, equivalence mapping, topic modeling, and a support vector-machine (SVM) in structured and unstructured EHR data. Scores were validated via blinded manual chart reviews (n=1.2K) of controls from a separate sample (n=20K). AUCs and negative and positive predictive values (NPVs and PPVs) were calculated to evaluate the models. There was a strong positive relationship between SVM-generated risk scores and undiagnosed dementia. BAs were more likely than WAs to have undiagnosed dementia per chart review, both overall (15.3% vs 9.5%) and among Veterans with >90th percentile cutoff scores (25.6% vs 15.3%). With chart reviews as the reference standard and varied cutoff scores, the BA model performed slightly better than the WA model (AUC=0.86 with NPV=0.98 and PPV=0.26 at >90th percentile cutoff vs AUC=0.77 with NPV=0.98 and PPV=0.15 at >90th). The AUCs, NPVs, and PPVs suggest that race-specific ML models can assist in the identification of undiagnosed dementia, particularly in BAs. Future studies should investigate implementing EHR-based risk scores in clinics that serve both BA and WA Veterans.

show abstract

Section: Discussionsupporting

confidence: 87%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records

Shao

Todd

Millard

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“… 28 Recent research has shown potential in the development of EMR-derived frailty indexes. 29 Further research efforts should be directed toward optimizing automated geriatric screening by using discrete data elements and emerging technologies such as machine learning 30 and natural language processing 31 to collect digital biomarkers housed within the EMR to drive geriatric evaluations and examine interventions that could improve outcomes in this vulnerable patient population.…”

Section: Discussionmentioning

confidence: 99%

Use of the electronic medical record to screen for high-risk geriatric patients in the emergency department

et al. 2023

View full text Add to dashboard Cite

Older adults with multimorbidities have the highest rate of emergency department (ED) usage. These patients are typically on numerous medications, may have underlying dementia, and often present with falls and delirium. Identifying these high-risk older adults for possible intervention is challenging in the ED setting since available screening methods are manual and resource-intensive. The objective is to study the electronic medical record (EMR) use for identifying high-risk older adults in ED. This feasibility study is conducted in an academic ED with 67 000 total and 24% geriatric (age ≥ 65 years) annual visits, American College of Emergency Physician (ACEP) accredited Level 1 Geriatric Emergency Department with an ED-based geriatric consultation program by incorporating criteria from existing manual geriatric screening instruments and the 4M framework into an automated EMR screen to identify high-risk geriatric patients. ED providers are then alerted by an EMR Best Practice Alert (BPA) if high-risk status is identified. Initial development and impact on geriatric ED consults are reported. During the study period, 7450 patient encounters occurred; 1836 (24.6%) encounters involved patients who were 65 years or older. A total of 1398 (76.1%) high-risk ED encounters resulted in BPA alerts using the EMR automated screen. BPA alerts resulted in 82 (5.9%) geriatric evaluations. We conclude that using the EMR to automate screening for older adults for high-risk geriatric conditions in the ED is feasible. An automated EMR screen with a BPA to ED providers identified a well-defined cohort of older patients appropriate for further ED geriatric evaluation.

show abstract

“…However, developing informatics-driven approaches, such as machine-learning algorithms, can overcome this. 50…”

Section: Discussionmentioning

confidence: 99%

Differential Effects of an Emergency Department-to-Home Care Transitions Intervention in an Older Adult Population

Green¹,

Nieser²,

Jacobsohn³

et al. 2023

Medical Care

Self Cite

View full text Add to dashboard Cite

Background: Older adults frequently return to the emergency department (ED) within 30 days of a visit. High-risk patients can differentially benefit from transitional care interventions. Latent class analysis (LCA) is a model-based method used to segment the population and test intervention effects by subgroup.Objectives: We aimed to identify latent classes within an older adult population from a randomized controlled trial evaluating the effectiveness of an ED-to-home transitional care program and test whether class membership modified the intervention effect.Research Design: Participants were randomized to receive the Care Transitions Intervention or usual care. Study staff collected outcomes data through medical record reviews and surveys. We performed LCA and logistic regression to evaluate the differential effects of the intervention by class membership.Subjects: Participants were ED patients (age 60 y and above) discharged to a community residence.Measures: Indicator variables for the LCA included clinically available and patient-reported data from the initial ED visit. Our primary outcome was ED revisits within 30 days. Secondary outcomes included ED revisits within 14 days, outpatient follow-up within 7 and 30 days, and self-management behaviors.Results: We interpreted 6 latent classes in this study population. Classes 1, 4, 5, and 6 showed a reduction in ED revisit rates with the intervention; classes 2 and 3 showed an increase in ED revisit rates. In class 5, we found evidence that the intervention increased outpatient follow-up within 7 and 30 days (odds ratio: 1.81, 95% CI: 1.13-2.91; odds ratio: 2.24, 95% CI: 1.25-4.03).Conclusions: Class membership modified the intervention effect. Population segmentation is an important step in evaluating a transitional care intervention.

show abstract

Machine learning‐assisted screening for cognitive impairment in the emergency department

Cited by 12 publications

References 26 publications

Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records

Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records

Use of the electronic medical record to screen for high-risk geriatric patients in the emergency department

Differential Effects of an Emergency Department-to-Home Care Transitions Intervention in an Older Adult Population

Contact Info

Product

Resources

About