Although evaluators often use an interrupted time series (ITS) design to test hypotheses about program effects, there are few empirical tests of the design's validity. We take a randomized experiment on an educational topic and compare its effects to those from a comparative ITS (CITS) design that uses the same treatment group as the experiment but a nonequivalent comparison group that is assessed at six time points before treatment. We estimate program effects with and without matching of the comparison schools, and we also systematically vary the number of pretest time points in the analysis. CITS designs produce impact estimates that are extremely close to the experimental benchmarks and, as implemented here, do so equally well with and without matching. Adding time points provides an advantage so long as the pretest trend differences in the treatment and comparison groups are correctly modeled. Otherwise, more time points can increase bias.
We explore the conditions under which short, comparative interrupted time-series (CITS) designs represent valid alternatives to randomized experiments in educational evaluations. To do so, we conduct three within-study comparisons, each of which uses a unique data set to test the validity of the CITS design by comparing its causal estimates to those from a randomized controlled trial (RCT) that shares the same treatment group. The degree of correspondence between RCT and CITS estimates depends on the observed pretest time trend differences and how they are modeled. Where the trend differences are clear and can be easily modeled, no bias results; where the trend differences are more volatile and cannot be easily modeled, the degree of correspondence is more mixed, and the best results come from matching comparison units on both pretest and demographic covariates.
Recognizing that cross-sectional data are often insufficient to address the identification problems associated with estimating the effect of government taxation or spending, economists engaged in public finance research often utilize longitudinal data that span the period over which a policy change occurred. As economic data have proliferated over the last decade, uses of the difference-indifferences design and its variations have become more numerous. Nevertheless, published research that invokes difference-indifferences commonly fails to present evidence and reasoning that enable the reader to properly evaluate the causal claims under investigation. In this paper, we examine the threats to internal validity that exist when using difference-indifferences for causal inference and review variations of the design that can be used to address these threats. Next, we survey the public finance literature in order to examine the ways that these threats are addressed in practice. We conclude by proposing a number of recommendations for researchers to consider as they implement difference-indifferences as an empirical strategy.
Much of the research on tax and expenditure limitations (TELs) focuses on the impact that limits have on the size of the public sector or the distribution of expenditures at the state and local levels. While these results shed light on the extent to which TELs succeed in reducing government spending, they do not have much to say about the impact of TELs on government budgeting or financial planning, despite the fact that voters support TELs in the hope of reducing government inefficiency (Courant, Gramlich, and Rubinfeld 1980;Ladd and Wilson 1982). This paper examines the effect of TELs on the stability of government revenues; sound tax policy entails controlling the volatility of revenues in order to plan more effectively for the future.
Despite the shortfalls in public employee pension funds, there is little known about the effect of fiscal institutions on pension funding. This paper focuses attention on the link between pension contributions and budget stabilization funds (BSFs) over the period 1997–2008. It employs the Blundell–Bond (1998) estimator in order to address the concern that the deposit and withdrawal rules that drive the management of BSFs may be endogenous to state pension contributions. Empirical results suggest that BSFs with strict deposit rules are associated with higher pension contributions, while strict withdrawal rules are associated with lower contributions.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.