Combining Matching and Synthetic Control to Trade off Biases from Extrapolation and Interpolation

Abstract

The synthetic control method is widely used in comparative case studies to adjust for differences in pre-treatment characteristics. A major attraction of the method is that it limits extrapolation bias that can occur when untreated units with different pre-treatment characteristics are combined using a traditional adjustment, such as a linear regression. Instead, the SC estimator is susceptible to interpolation bias because it uses a convex weighted average of the untreated units to create a synthetic untreated unit with pre-treatment characteristics similar to those of the treated unit. More traditional matching estimators exhibit the opposite behavior: they limit interpolation bias at the potential expense of extrapolation bias. We propose combining the matching and synthetic control estimators through model averaging to create an estimator called MASC. We show how to use a rolling-origin cross-validation procedure to train the MASC to resolve trade-o s between interpolation and extrapolation bias. We use a series of empirically-based placebo and Monte Carlo simulations to shed light on when the SC, matching, MASC and penalized SC estimators do (and do not) perform well. Then, we use the MASC re-examine the economic costs of conflicts and find evidence of larger effects than with SC.

None for the Road? Stricter Drink Driving Laws and Road Accidents

“…8 In the attempt to address the issue of noncommon trends between Scotland and the rest of Britain as persuasively as possible, limit the role played by unobserved local differences, and reduce the scope for omitted variables bias, we compare the results from three approaches, namely, models based on difference-in-differences, spatial regression discontinuity design, and synthetic control methods. We supplement both the difference-in-difference analysis using a linear panel event-study design following Freyaldenhoven, Hansen, and Shapiro (2019) and the synthetic control approach combining matching and synthetic control estimators through model averaging as proposed by Kellogg et al (2019). All estimates reveal that the lower Scottish limit had no impact on any type of road accident, from fatal crashes to collisions involving just slight injuries.…”

“…Other estimators instead, such as nearest-neighbor matching, have the opposite properties, that is, they curb interpolation bias but suffer from extrapolation bias, extrapolating too much when suitable untreated districts are unavailable. Kellogg et al (2019) suggest to optimize the strength of the two estimators and combine matching and synthetic control (MASC) procedures through model averaging. In the analysis below, we also employ this more recent approach.…”

The Inclusive Synthetic Control Method

“…Botosaru and Ferman (2019) discuss implications of not having perfect covariate balance and provide alternative assumptions under which SCM can still be used. Kellogg et al (2020) propose a model averaging method called "matching synthetic control estimator" that is a convex combination of the synthetic control and matching estimators. Their procedure gives weight to the synthetic control estimator that are proportional to the risk of having extrapolation bias.…”

Tactics for Design and Inference in Synthetic Control Studies: An Applied Example Using High-Dimensional Data

“…Convex weight restrictions allow for any amount of interpolation, no matter how extreme, and for no extrapolation, no matter how minor. Extreme interpolation can be just as undesireable as extreme extrapolation (King and Zeng, 2006;Kellogg et al, 2020).…”

“…A similar method has been proposed byKellogg et al (2020). In general this type of method can be thought of as a type of model averaging designed to reduce overfitting and the influence of noise(Athey et al, 2019).…”