Prospective Validation of an Electronic Health Record–Based, Real-Time Suicide Risk Model

Abstract

IMPORTANCE Numerous prognostic models of suicide risk have been published, but few have been implemented outside of integrated managed care systems. OBJECTIVE To evaluate performance of a suicide attempt risk prediction model implemented in a vendor-supplied electronic health record to predict subsequent (1) suicidal ideation and (2) suicide attempt. DESIGN, SETTING, AND PARTICIPANTS This observational cohort study evaluated implementation of a suicide attempt prediction model in live clinical systems without alerting. The cohort comprised patients seen for any reason in adult inpatient, emergency department, and ambulatory surgery settings at an academic medical center in the mid-South from June 2019 to April 2020. MAIN OUTCOMES AND MEASURES Primary measures assessed external, prospective, and concurrent validity. Manual medical record validation of coded suicide attempts confirmed incident behaviors with intent to die. Subgroup analyses were performed based on demographic characteristics, relevant clinical context/setting, and presence or absence of universal screening. Performance was evaluated using discrimination (number needed to screen, C statistics, positive/ negative predictive values) and calibration (Spiegelhalter z statistic). Recalibration was performed with logistic calibration. RESULTS The system generated 115 905 predictions for 77 973 patients (42 490 [54%] men, 35 404 [45%] women, 60 586 [78%] White, 12 620 [16%] Black). Numbers needed to screen in highest risk quantiles were 23 and 271 for suicidal ideation and attempt, respectively. Performance was maintained across demographic subgroups. Numbers needed to screen for suicide attempt by sex were 256 for men and 323 for women; and by race: 373, 176, and 407 for White, Black, and non-White/non-Black patients, respectively. Model C statistics were, across the health system: 0.836

Improving ascertainment of suicidal ideation and suicide attempt with natural language processing

“…Real-time clinical predictive studies are now underway and rely on structured data, e.g., diagnostic codes and problem lists, to track outcomes. 30 Because of inherent delays in diagnostic coding, systems that might review clinical text as it is entered into EHRs would enable faster and more accurate ascertainment to inform learning health systems for suicide prevention.…”

Applications of Clinical Informatics to Child Mental Health Care: a Call to Action to Bridge Practice and Training

“…Perhaps the most promising application of EHR-based risk prediction is suicide risk assessment. An increasing number of health systems now have implemented suicidal behavior risk prediction models [45], which utilize machine learning and data analytics to process data from millions of individuals and, in turn, deliver a personalized risk prediction of suicidal behavior for an individual child [46]. Here again there is potential for child psychiatrists to participate in the development and implementation of these predictive tools.…”

Integrating Social Determinants of Health to Precision Medicine through Digital Transformation: An Exploratory Roadmap

“…WHO’s position appears to be much broader in scope in targeting population health and not just in precision medicine. The literature on AI in healthcare in medical intervention has been primarily limited to precision medicine [ 11 , 12 , 13 ]. Reasons might be that improving precision medicine may achieve population health.…”