A machine learning framework supporting prospective clinical decisions applied to risk prediction in oncology

Abstract

We present a general framework for developing a machine learning (ML) tool that supports clinician assessment of patient risk using electronic health record-derived real-world data and apply the framework to a quality improvement use case in an oncology setting to identify patients at risk for a near-term (60 day) emergency department (ED) visit who could potentially be eligible for a home-based acute care program. Framework steps include defining clinical quality improvement goals, model development and validation, bias assessment, retrospective and prospective validation, and deployment in clinical workflow. In the retrospective analysis for the use case, 8% of patient encounters were associated with a high risk (pre-defined as predicted probability ≥20%) for a near-term ED visit by the patient. Positive predictive value (PPV) and negative predictive value (NPV) for future ED events was 26% and 91%, respectively. Odds ratio (OR) of ED visit (high- vs. low-risk) was 3.5 (95% CI: 3.4–3.5). The model appeared to be calibrated across racial, gender, and ethnic groups. In the prospective analysis, 10% of patients were classified as high risk, 76% of whom were confirmed by clinicians as eligible for home-based acute care. PPV and NPV for future ED events was 22% and 95%, respectively. OR of ED visit (high- vs. low-risk) was 5.4 (95% CI: 2.6–11.0). The proposed framework for an ML-based tool that supports clinician assessment of patient risk is a stepwise development approach; we successfully applied the framework to an ED visit risk prediction use case.

Risk Factors for Return to the Emergency Department and Readmission in Patients With Hospital-Diagnosed Advanced Lung Cancer

“…Although the analysis of hospital-level factors impacting readmissions could potentially provide valuable insight into factors an institution could target to reduce readmissions, it would require an alternative modeling approach. We believe that our focus on patient-level factors complements existing literature on system-level factors impacting readmission rates and aligns with other contemporary approaches to patient risk stratification 31–33. The emphasis on patient-level factors in the current model best supports translation of these study findings into a clinical risk stratification tool that can be used to identify individual high-risk patients.…”

“…We believe that our focus on patient-level factors complements existing literature on system-level factors impacting readmission rates and aligns with other contemporary approaches to patient risk stratification. [31][32][33] The emphasis on patient-level factors in the current model best supports translation of these study findings into a clinical risk stratification tool that can be used to identify individual high-risk patients.…”

Predicting Blood Pressure After Nitroglycerin Infusion Dose Titration in Critical Care Units

“…These results are expected because prior and average values of the prediction target (in our case, systolic blood pressure) are known to be informative for modeling tasks. 19,20 It should be noted that permutation importance methods reveal how important particular features are to a model, not the predictive feature itself. 11 As such, although the features related to nitroglycerin dose titration seemed to contribute only minimally for the best-performing model in our analyses, it does not mean this information is not helpful at all for the task of systolic blood pressure prediction during nitroglycerin infusion.…”

Predicting Acute Care Events Among Patients Initiating Chemotherapy: A Practice-Based Validation and Adaptation of the PROACCT Model

“…Head and neck 83 ( The resulting adaptations, which sought to facilitate model integration within EHRs by easing the incorporation of new therapeutic agents, yielded similar performance to the original and other previously published predictive models in terms of positive predictive value and C statistic. 8,9,11,15 As cancer care organizations seek to adopt population management approaches, effective risk stratification becomes a key mechanism to maximize improvement efforts. Accurately and efficiently identifying groups of patients at high risk for complications during and after anticancer treatment allows targeted service delivery that could mitigate risk.…”