Methods for estimating confidence intervals in interrupted time series analyses of health interventions

Abstract

Objective Interrupted time series is a strong quasi-experimental research design that is increasingly applied to estimate the effects of health services and policy interventions. We describe and illustrate two methods for estimating confidence intervals around absolute and relative changes in outcomes calculated from segmented regression parameter estimates. Study Design and Setting We used multivariate delta and bootstrapping methods to construct confidence intervals around relative changes in level and trend and around absolute changes in outcome based on segmented linear regression analyses of time series data corrected for auto-correlated errors. Results Using previously published time series data, we estimated confidence intervals around the effect of prescription alerts for interacting medications with warfarin on the rate of prescriptions per 10,000 warfarin users per month. Both the multivariate delta method and the bootstrapping method produced similar results. Conclusion The bootstrapping method is preferred for calculating confidence intervals of relative changes in outcomes of time series studies since it does not require large sample sizes when parameter estimates are obtained correctly from the model. Caution is needed when sample size is small.

A population-based analysis of incentive payments to primary care physicians for the care of patients with complex disease

“…We estimated the absolute change in each outcome as the difference between the counterfactual (the preintervention trend projected forward) and the observed postintervention values, averaged across the 24 months postintervention, with bootstrapped 95% confidence intervals (CIs). 21 We conducted analyses in R 3.0.3 using the nlme and car packages.…”

Anti-Osteoporosis Medication Prescriptions and Incidence of Subsequent Fracture Among Primary Hip Fracture Patients in England and Wales: An Interrupted Time-Series Analysis

“…For ease of interpretation, we expressed regression coefficients for level and slope in the form of a single estimate of absolute change between estimated postintervention values and their counterfactual values (10,13) ; ie, estimates for the same time point but based on preintervention level and trend only. We used the October 2007 to March 2008 time point because the end of March 2008 was the middle of the postintervention period (for the fracture time series).…”

Evaluation of an intravenous preparation information system for improving the reconstitution and dilution process

“…The dilution errors might be higher than 28.6, 24.7, and 18.4% in baseline, IPIS_R, and IPIS_RD periods, respectively, and those error rates might be similar or even higher than IV preparation errors reported in other studies (20,26, and 49%, respectively) [1][2][3]. Because of the design of this retrospective study, we could not determine whether nurses selected the correct reconstitution and dilution solutions without a physician's order.…”