For EHR-derived data to yield reliable real-world evidence, it needs to be of known and sufficiently high quality. Considering the impact of mortality data completeness on survival endpoints, we highlight the importance of data quality assessment and advocate benchmarking to the NDI.

PURPOSELarge, generalizable real-world data can enhance traditional clinical trial results. The current study evaluates reliability, clinical relevance, and large-scale feasibility for a previously documented method with which to characterize cancer progression outcomes in advanced non–small-cell lung cancer from electronic health record (EHR) data.METHODSPatients who were diagnosed with advanced non–small-cell lung cancer between January 1, 2011, and February 28, 2018, with two or more EHR-documented visits and one or more systemic therapy line initiated were identified in Flatiron Health’s longitudinal EHR-derived database. After institutional review board approval, we retrospectively characterized real-world progression (rwP) dates, with a random duplicate sample to ascertain interabstractor agreement. We calculated real-world progression-free survival, real-world time to progression, real-world time to next treatment, and overall survival (OS) using the Kaplan-Meier method (index date was the date of first-line therapy initiation), and correlations between OS and other end points were assessed at the patient level (Spearman’s ρ).RESULTSOf 30,276 eligible patients,16,606 (55%) had one or more rwP event. Of these patients, 11,366 (68%) had subsequent death, treatment discontinuation, or new treatment initiation. Correlation of real-world progression-free survival with OS was moderate to high (Spearman’s ρ, 0.76; 95% CI, 0.75 to 0.77; evaluable patients, n = 20,020), and for real-world time to progression correlation with OS was lower (Spearman’s ρ, 0.69; 95% CI, 0.68 to 0.70; evaluable patients, n = 11,902). Interabstractor agreement on rwP occurrence was 0.94 (duplicate sample, n = 1,065) and on rwP date 0.85 (95% CI, 0.81 to 0.89; evaluable patients n = 358 [patients with two independent event captures within 30 days]). Median rwP abstraction time from individual EHRs was 18.0 minutes (interquartile range, 9.7 to 34.4 minutes).CONCLUSIONWe demonstrated that rwP-based end points correlate with OS, and that rwP curation from a large, contemporary EHR data set can be reliable, clinically relevant, and feasible on a large scale.

IntroductionReal-world evidence derived from electronic health records (EHRs) is increasingly recognized as a supplement to evidence generated from traditional clinical trials. In oncology, tumor-based Response Evaluation Criteria in Solid Tumors (RECIST) endpoints are standard clinical trial metrics. The best approach for collecting similar endpoints from EHRs remains unknown. We evaluated the feasibility of a RECIST-based methodology to assess EHR-derived real-world progression (rwP) and explored non-RECIST-based approaches.MethodsIn this retrospective study, cohorts were randomly selected from Flatiron Health’s database of de-identified patient-level EHR data in advanced non-small cell lung cancer. A RECIST-based approach tested for feasibility (N = 26). Three non-RECIST approaches were tested for feasibility, reliability, and validity (N = 200): (1) radiology-anchored, (2) clinician-anchored, and (3) combined. Qualitative and quantitative methods were used.ResultsA RECIST-based approach was not feasible: cancer progression could be ascertained for 23% (6/26 patients). Radiology- and clinician-anchored approaches identified at least one rwP event for 87% (173/200 patients). rwP dates matched 90% of the time. In 72% of patients (124/173), the first clinician-anchored rwP event was accompanied by a downstream event (e.g., treatment change); the association was slightly lower for the radiology-anchored approach (67%; 121/180). Median overall survival (OS) was 17 months [95% confidence interval (CI) 14, 19]. Median real-world progression-free survival (rwPFS) was 5.5 months (95% CI 4.6, 6.3) and 4.9 months (95% CI 4.2, 5.6) for clinician-anchored and radiology-anchored approaches, respectively. Correlations between rwPFS and OS were similar across approaches (Spearman’s rho 0.65–0.66). Abstractors preferred the clinician-anchored approach as it provided more comprehensive context.ConclusionsRECIST cannot adequately assess cancer progression in EHR-derived data because of missing data and lack of clarity in radiology reports. We found a clinician-anchored approach supported by radiology report data to be the optimal, and most practical, method for characterizing tumor-based endpoints from EHR-sourced data.FundingFlatiron Health Inc., which is an independent subsidiary of the Roche group.Electronic supplementary materialThe online version of this article (10.1007/s12325-019-00970-1) contains supplementary material, which is available to authorized users.

Real-world evidence derived from electronic health records (EHRs) is increasingly recognized as a supplement to evidence generated from traditional clinical trials. In oncology, tumor-based Response Evaluation Criteria in Solid Tumors (RECIST) endpoints are collected in clinical trials. The best approach for collecting similar endpoints from EHRs remains unknown.We evaluated the feasibility of a traditional RECIST-based methodology to assess EHR-derived real-world progression (rwP) and explored non-RECIST-based approaches. In this retrospective study, cohorts were randomly selected from Flatiron Health's database of patient-level EHR data in advanced non-small cell lung cancer. A RECIST-based approach was tested for feasibility (N=26). Three non-RECIST abstraction approaches were tested for feasibility, reliability, and validity (N=200): (1) radiology-anchored, (2) clinician-anchored, and (3) combined.RECIST-based cancer progression could be ascertained from the EHRs of 23% of patients (6/26).In 87% of patients (173/200), at least one rwP event was identified using both the radiologyand clinician-anchored approaches. rwP dates matched 90% of the time. In 72% of patients (124/173), the first clinician-anchored rwP event was accompanied by a downstream event (e.g., treatment change); the association was slightly lower for the radiology-anchored approach (67%; 121/180). Median overall survival (OS) was 17 months (95% confidence interval [CI]: 14, 19). Median real-world progression-free survival (rwPFS) was 5.5 (95% CI: 4.6, 6.3) and 4.9 months (95% CI: 4.2, 5.6) for clinician-anchored and radiology-anchored approaches, respectively. Correlations between rwPFS and OS were similar across approaches (Spearman's rho: 0.65-0.66). Abstractors preferred the clinician-anchored approach as it provided more

Amid growing excitement for immune checkpoint inhibitors of programmed death protein 1 (anti-PD1 agents), little is known about whether race- or sex-based disparities exist in their use. In this observational study, we constructed a large and mostly community-based cohort of patients with advanced stage cancers, including melanoma, non-small cell lung cancer (NSCLC), and renal cell carcinoma, to compare the odds of receiving systemic treatment with or without anti-PD1 agents by race and by sex. In multivariable models that adjusted for age, stage, and number of prior anticancer therapies, we found no significant race-based disparities in anti-PD1 treatment. However, among patients with NSCLC, males had significantly higher odds of receiving anti-PD1 treatment compared with females (odds ratio 1.13, 95% confidence interval 1.02-1.24, = .02). This finding suggests that as anti-PD1 agents enter the market to transform patient care, it will be critical to monitor for disparities in the use of these drugs.