Validating a Patient-Reported Outcomes–Derived Algorithm for Classifying Symptom Complexity Levels Among Patients With Cancer

Abstract

Background: The patient-reported outcomes (PROs) symptom complexity algorithm, derived from self-reported symptom scores using the Edmonton Symptom Assessment System and concerns indicated on the Canadian Problem Checklist, has not been validated extensively. Methods: This is a retrospective chart review study using data from the Alberta Cancer Registry and electronic medical records from Alberta Health Services. The sample includes patients with cancer who visited a cancer facility in Alberta, Canada, from February 2016 through November 2017 (n=1,466). Results: The effect size (d=1.2) indicates that the magnitude of difference in health status between the severe- and low-complexity groups is large. The symptom complexity algorithm effectively classified subgroups of patients with cancer with distinct health status. Using Karnofsky performance status, the algorithm shows a sensitivity of 70.3%, specificity of 84.1%, positive predictive value of 79.1%, negative predictive value of 76.7%, and accuracy of 77.7%. An area under the receiver operating characteristic of 0.824 was found for the complexity algorithm, which is generally regarded as good, This same finding was also regarded as superior to the alternative algorithm generated by 2-step cluster analysis (area under the curve, 0.721). Conclusions: The validity of the PRO-derived symptom complexity algorithm is established in this study. The algorithm demonstrated satisfactory accuracy against a clinician-driven complexity assessment and a strong correlation with the known group analysis. Furthermore, the algorithm showed a higher screening capacity compared with the algorithm generated from 2-step cluster analysis, reinforcing the importance of contextualization when classifying patients’ symptoms, rather than purely relying on statistical outcomes. The algorithm carries importance in clinical settings, acting as a symptom complexity flag, helping healthcare teams identify which patients may need more timely, targeted, and individualized patient symptom management.

The Use of Patient-Reported Outcome Measures in Phase I Oncology Clinical Trials

“…There can also be a perception that PRO endpoints are not as important as clinical endpoints, meaning that researchers may not follow up as carefully with patients. Healthcare providers may be resistant to using PROs or find the number of PRO data points overwhelming [37]. Validation of PRO measures is important in demonstrating PRO suitability and may assist in increasing healthcare providers' acceptance of PRO data.…”

Symptom Burden and Complexity in the Last 12 Months of Life among Cancer Patients Choosing Medical Assistance in Dying (MAID) in Alberta, Canada

“…It includes two standardized measures: the Edmonton Symptom Assessment System-Revised (ESAS-r), and the Canadian Problem Checklist (CPC) [18]. Together, the symptoms and concerns reported by the patient are used to generate a symptom complexity score from a unique validated algorithm [19]. The PROs questionnaire is used with all patients, including those with advanced cancer, and as such, the symptoms and concerns of specific patient cohorts can be tracked over time.…”

“…The ESAS-r and CPC results were used to determine a symptom complexity score for each PROs questionnaire, based on an original validated algorithm created within CCA [19]. The algorithm considers the unique combination of symptoms and concerns a patient has identified on a single questionnaire and assigns a symptom complexity score (low/green, moderate/yellow or high/red) for each encounter.…”

Using Autoregressive Integrated Moving Average (ARIMA) Modelling to Forecast Symptom Complexity in an Ambulatory Oncology Clinic: Harnessing Predictive Analytics and Patient-Reported Outcomes

“…This clinician-facing summary score was designed to support clinical teams in their efforts to tailor their time allocation and approach to each patient. The algorithm has shown satisfactory accuracy, good discrimination, and calibration in classifying symptom complexity levels among cancer patients [21].…”