<p>Sample size and power considerations for ordinary least squares interrupted time series analysis: a simulation study</p>

Abstract

Interrupted time series (ITS) analysis is being increasingly used in epidemiology. Despite its growing popularity, there is a scarcity of guidance on power and sample size considerations within the ITS framework. Our aim of this study was to assess the statistical power to detect an intervention effect under various real-life ITS scenarios. ITS datasets were created using Monte Carlo simulations to generate cumulative incidence (outcome) values over time. We generated 1,000 datasets per scenario, varying the number of time points, average sample size per time point, average relative reduction post intervention, location of intervention in the time series, and reduction mediated via a 1) slope change and 2) step change. Performance measures included power and percentage bias. We found that sample size per time point had a large impact on power. Even in scenarios with 12 pre-intervention and 12 post-intervention time points with moderate intervention effect sizes, most analyses were underpowered if the sample size per time point was low. We conclude that various factors need to be collectively considered to ensure adequate power for an ITS study. We demonstrate a means of providing insight into underlying sample size requirements in ordinary least squares (OLS) ITS analysis of cumulative incidence measures, based on prespecified parameters and have developed Stata code to estimate this.

Trends in Family Violence Are Not Causally Associated with COVID-19 Stay-at-Home Orders: a Commentary on Piquero et al.

“…The study by Piquero et al ( 2020 ) compared family violence rates during the 83 days before the stay at home order was enacted (Code Section 418.10, 2020 ) to rates for just more than one month ( n = 35 days) after the stay-at-home order using secondary data from the Dallas Police Department (DPD). Epidemiological research suggests that even the most robust statistical analyses used to assess the outcomes of public policy changes are insufficiently powered to detect effect sizes of −/+15% (Hawley, Ali, Berencsi, Judge and Prieto-Alhambra, 2019 ). According to Hawley et al ( 2019 ), thousands of time points would be necessary to detect an effect size of 15%, indicating that the effect sizes in Piquero et al ( 2020 ) were unlikely to ever be statistically significant.…”

“…Epidemiological research suggests that even the most robust statistical analyses used to assess the outcomes of public policy changes are insufficiently powered to detect effect sizes of −/+15% (Hawley, Ali, Berencsi, Judge and Prieto-Alhambra, 2019 ). According to Hawley et al ( 2019 ), thousands of time points would be necessary to detect an effect size of 15%, indicating that the effect sizes in Piquero et al ( 2020 ) were unlikely to ever be statistically significant.…”

Developmental Changes in the Philippine Health System: Accomplishments, Successes and Challenges

“…For health management/development, there may be other features and challenges not fully captured in the summary; however, if such information becomes available in the near future, there is a need to run a re-analysis. While the current number of observations before and after 2008 were beyond the minimum number of observations required to do an ITS [18], extending the number of observations may provide a more robust insight on the estimates. In this study, the quantitative analyses relied on the available data sources as well as grey literature in assessing the progress of the Philippine Health System.…”

“…If the number of continuous data points for each period was less than five or if the data points were not continuous, the mean of the data points was utilized instead; as shown in Figure 1 below. While the minimum number of observations to carry out ITS was 3 [18], in this study, variables with at least 5 temporally consecutive observations before and after 2008 were analyzed (for ITS). On the other hand, the mean was taken for variables which were either (a) non-consecutive 5 observations or (b) less than 5 observations.…”

Flawed conclusions on the Västerbotten Intervention Program by San Sebastian et .al

“…ITS is designed for evaluating interventions where 1) there is a clearly defined cut point corresponding to the time the intervention is implemented [16] and 2) the effects are expected to be felt relatively quickly after implementation, or after a clearly defined lag time [17]. Neither of these two conditions apply to the VIP.…”