Building a Clinically Relevant Risk Model: Predicting Risk of a Potentially Preventable Acute Care Visit for Patients Starting Antineoplastic Treatment

Abstract

PURPOSE To create a risk prediction model that identifies patients at high risk for a potentially preventable acute care visit (PPACV). PATIENTS AND METHODS We developed a risk model that used electronic medical record data from initial visit to first antineoplastic administration for new patients at Memorial Sloan Kettering Cancer Center from January 2014 to September 2018. The final time-weighted least absolute shrinkage and selection operator model was chosen on the basis of clinical and statistical significance. The model was refined to predict risk on the basis of 270 clinically relevant data features spanning sociodemographics, malignancy and treatment characteristics, laboratory results, medical and social history, medications, and prior acute care encounters. The binary dependent variable was occurrence of a PPACV within the first 6 months of treatment. There were 8,067 observations for new-start antineoplastic therapy in our training set, 1,211 in the validation set, and 1,294 in the testing set. RESULTS A total of 3,727 patients experienced a PPACV within 6 months of treatment start. Specific features that determined risk were surfaced in a web application, riskExplorer, to enable clinician review of patient-specific risk. The positive predictive value of a PPACV among patients in the top quartile of model risk was 42%. This quartile accounted for 35% of patients with PPACVs and 51% of potentially preventable inpatient bed days. The model C-statistic was 0.65. CONCLUSION Our clinically relevant model identified the patients responsible for 35% of PPACVs and more than half of the inpatient beds used by the cohort. Additional research is needed to determine whether targeting these high-risk patients with symptom management interventions could improve care delivery by reducing PPACVs.

InSight Care Pilot Program: Redefining Seeing a Patient

“…16 Predictive analytic models can focus resources on those most in need; however, few have been implemented in clinical practice. 11 We found that risk stratification models must work in collaboration with the clinician. Most patients were enrolled in the program on the basis of clinical risk criteria and not their model-predicted high-risk status.…”

“…Eligible patients (1) were 18 years of age or older; (2) had a diagnosis of a solid tumor or lymphoma; (3) were starting intravenous antineoplastic therapy, including a cytotoxic, immunologic, or biologic agent, that was initial therapy or had been at least 6 months since their last treatment; (4) had access to a smartphone, tablet, or computer; (5) were enrolled in the patient portal; and (6) were identified by our risk stratification model or clinical criteria as at high risk of a PPACV. The risk stratification model, described elsewhere, 11 used data from the electronic medical record (EMR) to prospectively estimate the risk of a PPACV within the next 6 months for patients starting intravenous therapy. The top quartile of patients identified by this model were categorized as "high risk" and were offered enrollment in the program.…”

System for High-Intensity Evaluation During Radiation Therapy (SHIELD-RT): A Prospective Randomized Study of Machine Learning–Directed Clinical Evaluations During Radiation and Chemoradiation

“…This strategy may also be appropriate in other settings, including for patients undergoing systemic therapy, which has been the subject of many prediction models. 9,10,27,28 We additionally evaluated reasons for visits to determine if certain types of acute care were preventable. CMS-designated preventable diagnoses during RT were less common reasons for acute care in the intervention arm, though more data may be required to understand which acute care visits are truly preventable.…”

“…We previously developed a ML algorithm utilizing patient electronic health record (EHR) data, which retrospectively demonstrated strong predictive ability to identify patients at high risk for acute care, 8 comparing favorably to other models for this complex problem. 9,10 Despite a growing body of retrospective medical ML literature, 11,12 prospective evaluation remains tremendously limited, primarily to diagnostic fields. [13][14][15][16] Despite the need for prospective interventional trials, 17 there have been few, including a recent trial effectively predicting intraoperative hypotension utilizing arterial catheter sensor data.…”

Machine Learning Applied to Electronic Health Records: Identification of Chemotherapy Patients at High Risk for Preventable Emergency Department Visits and Hospital Admissions

“…13 ML applied to EHRs has been used specifically to tackle the issue of identifying patients with cancer at risk for ACU. [14][15][16][17] Although these studies advance our knowledge in the field, they were limited to a small number of available variables in the EHR, used logistic regression instead of more robust AI models, and/or did not use OP-35 criteria to determine preventable ACU.…”