Development of Machine Learning Algorithms Incorporating Electronic Health Record Data, Patient-Reported Outcomes, or Both to Predict Mortality for Outpatients With Cancer

Abstract

PURPOSE Machine learning (ML) algorithms that incorporate routinely collected patient-reported outcomes (PROs) alongside electronic health record (EHR) variables may improve prediction of short-term mortality and facilitate earlier supportive and palliative care for patients with cancer. METHODS We trained and validated two-phase ML algorithms that incorporated standard PRO assessments alongside approximately 200 routinely collected EHR variables, among patients with medical oncology encounters at a tertiary academic oncology and a community oncology practice. RESULTS Among 12,350 patients, 5,870 (47.5%) completed PRO assessments. Compared with EHR- and PRO-only algorithms, the EHR + PRO model improved predictive performance in both tertiary oncology (EHR + PRO v EHR v PRO: area under the curve [AUC] 0.86 [0.85-0.87] v 0.82 [0.81-0.83] v 0.74 [0.74-0.74]) and community oncology (area under the curve 0.89 [0.88-0.90] v 0.86 [0.85-0.88] v 0.77 [0.76-0.79]) practices. CONCLUSION Routinely collected PROs contain added prognostic information not captured by an EHR-based ML mortality risk algorithm. Augmenting an EHR-based algorithm with PROs resulted in a more accurate and clinically relevant model, which can facilitate earlier and targeted supportive care for patients with cancer.

Recent advances in artificial intelligence applications for supportive and palliative care in cancer patients

“…We found that the most common use of AI for predictive modeling in SPC was focused on mortality. Many studies in Table 1 and approximately half of all studies included in total focused on predicting mortality as a clinical outcome, which includes predicting short-term mortality risk 14▪,15,18▪,24▪ and survival over a longer horizon 19–21,22▪▪,23▪▪,25,29. Mortality risk and survival time were both usually predicted using machine learning (ML) models that analyze various patient factors such as clinical parameters, changes during treatment, and symptoms 14▪,15,18▪,20,21,22▪▪,23▪▪,24▪,25.…”

“…Many studies in Table 1 and approximately half of all studies included in total focused on predicting mortality as a clinical outcome, which includes predicting short-term mortality risk 14▪,15,18▪,24▪ and survival over a longer horizon 19–21,22▪▪,23▪▪,25,29. Mortality risk and survival time were both usually predicted using machine learning (ML) models that analyze various patient factors such as clinical parameters, changes during treatment, and symptoms 14▪,15,18▪,20,21,22▪▪,23▪▪,24▪,25. The accuracy of ML and deep learning (DL) models is typically evaluated by their area under the curve (AUC) value, which measures the accuracy of predictions and a model’s discriminative ability where 1.0 represents the highest possible AUC score indicating perfect discrimination 11,39.…”

An overview and a roadmap for artificial intelligence in hematology and oncology

“…Electronic health records (EHR) are at the core of documenting any patient contact in oncology, and also integrate multimodal data related to the diagnosis of cancer and biomarkers for precision oncology (Parikh et al 2022;Morin et al 2021;Araki et al 2022). Much EHR data are unstructured or just loosely structured, making it difficult to mine historically.…”

“…While EHR comprises mostly data generated by healthcare staff, the patient perspective can be underrepresented. This gap is filled by patient-reported outcome-and experience measures (PROMs, PREMs) including a data source of the patient's perspective which is increasingly being acknowledged in oncology as clinically relevant outcome measures in clinical trials and in certification processes evaluating health care in cancer centers (Parikh et al 2022). EHR and PROM/PREM data are part of the loosely defined category of "real-world data" (RWD).…”

On the importance of interpretable machine learning predictions to inform clinical decision making in oncology

“…For oncology practice, ML-based tools are often developed and used to support high-stakes decisions, such as diagnosis (16,17), advance care planning communication (18,19), and treatment selection (20). Providing only predictions is not enough to solve all problems for these tasks, and a model should provide explanations concerning its decision-making to allow human reasoning and preventative actions (11).…”