Local Instrumental Variable Methods to Address Confounding and Heterogeneity when Using Electronic Health Records: An Application to Emergency Surgery

Abstract

Background Electronic health records (EHRs) offer opportunities for comparative effectiveness research to inform decision making. However, to provide useful evidence, these studies must address confounding and treatment effect heterogeneity according to unmeasured prognostic factors. Local instrumental variable (LIV) methods can help studies address these challenges, but have yet to be applied to EHR data. This article critically examines a LIV approach to evaluate the cost-effectiveness of emergency surgery (ES) for common acute conditions from EHRs. Methods This article uses hospital episodes statistics (HES) data for emergency hospital admissions with acute appendicitis, diverticular disease, and abdominal wall hernia to 175 acute hospitals in England from 2010 to 2019. For each emergency admission, the instrumental variable for ES receipt was each hospital’s ES rate in the year preceding the emergency admission. The LIV approach provided individual-level estimates of the incremental quality-adjusted life-years, costs and net monetary benefit of ES, which were aggregated to the overall population and subpopulations of interest, and contrasted with those from traditional IV and risk-adjustment approaches. Results The study included 268,144 (appendicitis), 138,869 (diverticular disease), and 106,432 (hernia) patients. The instrument was found to be strong and to minimize covariate imbalance. For diverticular disease, the results differed by method; although the traditional approaches reported that, overall, ES was not cost-effective, the LIV approach reported that ES was cost-effective but with wide statistical uncertainty. For all 3 conditions, the LIV approach found heterogeneity in the cost-effectiveness estimates across population subgroups: in particular, ES was not cost-effective for patients with severe levels of frailty. Conclusions EHRs can be combined with LIV methods to provide evidence on the cost-effectiveness of routinely provided interventions, while fully recognizing heterogeneity. Highlights This article addresses the confounding and heterogeneity that arise when assessing the comparative effectiveness from electronic health records (EHR) data, by applying a local instrumental variable (LIV) approach to evaluate the cost-effectiveness of emergency surgery (ES) versus alternative strategies, for patients with common acute conditions (appendicitis, diverticular disease, and abdominal wall hernia). The instrumental variable, the hospital’s tendency to operate, was found to be strongly associated with ES receipt and to minimize imbalances in baseline characteristics between the comparison groups. The LIV approach found that, for each condition, there was heterogeneity in the estimates of cost-effectiveness according to baseline characteristics. The study illustrates how an LIV approach can be applied to EHR data to provide cost-effectiveness estimates that recognize heterogeneity and can be used to inform decision making as well as to generate hypotheses for further research.

Reporting of Observational Studies Explicitly Aiming to Emulate Randomized Trials

“…We retrieved 3133 unique records, of which 200 were included in the review (Figure 1). All reasons for excluding records after full-text review are given in eAppendix 6 in Supplement 1.…”

“… The included studies spanned 26 fields of medicine, predominately infectious disease (43 [22%]; 27 [14%] on COVID-19), cardiology (30 [15%]), and oncology (30 [15%]) . One hundred and thirty-one studies (66%)…”

“…One-hundred and fifteen studies (58%) completely reported how the target trial protocol was emulated. Eighty-seven studies (44%) provided both the protocol of the target trial and described how it was emulated (Table 2).…”

Transparency and Rigor: Target Trial Emulation Aims to Achieve Both

“…However, other estimation approaches that do not rely on this assumption, such as instrumental variables and difference in differences, are also used in the target trial emulation literature (e.g., in Refs. 11 and 12). Of note, with these estimation methods, the no unmeasured confounding assumption is replaced by other unverifiable assumptions (e.g., exclusion restriction with monotonicity) which may be more plausible in certain settings but not others.…”

Filling the Gap Between Potential and Actual Usefulness of Electronic Health Record (EHR) Data as Patient-Level Evidence

“…Their research has the merit of casting a light on the ability and limitations of a validated Instrumental Variable approach to detect subtle cost-effectiveness patterns from an inherently complex EHR data structure. 5…”