Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets

Firpo, Sérgio; Possebom, Vitor

doi:10.1515/jci-2016-0026

Cited by 159 publications

(157 citation statements)

References 67 publications

Supporting

Mentioning

156

Contrasting

Order By: Relevance

“…We can also calculate confidence intervals by inverting the permutation test. Specifically, as suggested in Firpo and Possebom (), we rerun the permutation method where the posttreatment outcome for Colorado is transformed by a particular α *. The 90% confidence interval is then defined at the set of α * where we cannot reject the null hypothesis of Colorado being significantly different than more than 90% of placebo treatments.…”

Section: Methodology and Datamentioning

confidence: 99%

See 1 more Smart Citation

The Pot Rush: Is Legalized Marijuana a Positive Local Amenity?

Zambiasi

Stillman

2019

Economic Inquiry

View full text Add to dashboard Cite

This paper examines the amenity value of legalized marijuana by analyzing the impact of marijuana legalization on migration to Colorado. Colorado is the pioneering state in this area having legalized medical marijuana in 2000 and recreational marijuana in 2012. We test whether potential migrants to Colorado view legalized marijuana as a positive or negative local amenity. We use the synthetic control methodology to examine in‐ and out‐migration to/from Colorado versus migration to/from counterfactual versions of Colorado that have not legalized marijuana. We find strong evidence that potential migrants view legalized marijuana as a positive amenity with in‐migration significantly higher in Colorado compared with synthetic‐Colorado after the writing of the Ogden memo in 2009 that effectively allowed state laws already in place to be activated, and additionally after marijuana was legalized in 2013 for recreational use. When we employ permutation methods to assess the statistical likelihood of our results given our sample, we find that Colorado is a clear and significant outlier. We find no evidence for changes in out‐migration from Colorado suggesting that marijuana legalization did not change the equilibrium for individuals already living in the state. (JEL I18, R23, K42, C22)

show abstract

Section: Methodology and Datamentioning

confidence: 99%

“…We can also calculate confidence intervals by inverting the permutation test. Specifically, as suggested in Firpo and Possebom (2017), we rerun the permutation method where the 8. Abadie, Diamond, and Hainmueller (2010) provide more details about the synthetic control methodology.…”

Section: A the Synthetic Control Methodsmentioning

confidence: 99%

The Pot Rush: Is Legalized Marijuana a Positive Local Amenity?

Zambiasi

Stillman

2019

Economic Inquiry

View full text Add to dashboard Cite

show abstract

“…While are results are for permutations across the time series dimension, our general results on exact and approximate permutation inference (Propositions 1 and 2) also apply to permutations across units (subject to switching indices). This provides a rigorous formal justification for the inference procedure of Abadie et al (2010Abadie et al ( , 2015 under a set of sufficient conditions, which differ substantially from existing ones (e.g., Firpo and Possebom, 2017). We will justify our approach for a great variety of models to build counterfactual proxies for outcomes in the absence of the policy intervention, including many popular synthetic control, panel data, and fused time series panel models.…”

Section: Related Literaturementioning

confidence: 96%

“…One part of the literature considers a finite population approach, which relies on the assumption that potential outcomes are fixed but a priori unknown and that, conditional on observables, the treatment assignment is random (Firpo and Possebom, 2017). These assumptions justify the application of permutation tests similar to Fisher (1935)'s randomization test.…”

Section: Related Literaturementioning

confidence: 99%

“…For instance, Abadie et al (2010Abadie et al ( , 2015 permute which unit is assigned to the treatment and then compare the actual treatment effect estimates to the permutation distribution. 3 Firpo and Possebom (2017) and Ferman and Pinto (2017b) provide a comprehensive discussion of the theoretical aspects of such testing procedures. While finite population permutation approaches have traditionally been employed in conjunction with synthetic control methods, they can also be applied to a broader class of methods including difference-in-differences approaches, elastic net, and best subset selection, see, e.g., Doudchenko and Imbens (2016).…”

Section: Related Literaturementioning

confidence: 99%

See 1 more Smart Citation

An exact and robust conformal inference method for counterfactual and synthetic controls

Chernozhukov

Wüthrich

Zhu

2017

View full text Add to dashboard Cite

This paper introduces new inference methods for counterfactual and synthetic control methods for evaluating policy effects. Our inference methods work in conjunction with many modern and classical methods for estimating the counterfactual mean outcome in the absence of a policy intervention. Specifically, our methods work together with the difference-in-difference, canonical synthetic control, constrained and penalized regression methods for synthetic control, factor/matrix completion models for panel data, interactive fixed effects panel models, time series models, as well as fused time series panel data models. The proposed method has a double justification. (i) If the residuals from estimating the counterfactuals are exchangeable as implied, for example, by i.i.d. data, our procedure achieves exact finite sample size control without any assumption on the specific approach used to estimate the counterfactuals. (ii) If the data exhibit dynamics and serial dependence, our inference procedure achieves approximate uniform size control under weak and easy-to-verify conditions on the method used to estimate the counterfactual. We verify these condition for representative methods from each group listed above. Simulation experiments demonstrate the usefulness of our approach in finite samples. We apply our method to re-evaluate the causal effect of election day registration (EDR) laws on voter turnout in the United States.

show abstract

Cherry Picking with Synthetic Controls

Ferman¹,

Pinto²,

Possebom³

2020

J Policy Anal Manage

Self Cite

178

150

View full text Add to dashboard Cite

We evaluate whether a lack of guidance on how to choose the matching variables used in the Synthetic Control (SC) estimator creates specification-searching opportunities. We provide theoretical results showing that specification-searching opportunities are asymptotically irrelevant if we restrict to a subset of SC specifications. However, based on Monte Carlo simulations and simulations with real datasets, we show significant room for specification searching when the number of pre-treatment periods is in line with common SC applications, and when alternative specifications commonly used in SC applications are also considered. This suggests that such lack of guidance generates a substantial level of discretion in the choice of the comparison units in SC applications, undermining one of the advantages of the method. We provide recommendations to limit the possibilities for specification searching in the SC method. Finally, we analyze the possibilities for specification searching and provide our recommendations in a series of empirical applications. C 2020 by the Association for Public Policy Analysis and Management on behalf of the Association for Public Policy Analysis and Management variables, the choice of how to split the pre-treatment periods into training and validation periods, and even the choice of software and data-sorting criteria (see Klößner et al., 2017, for details on this last point). Therefore, our results should be seen as a lower bound on the possibilities for specification searching in SC applications. We focus on the choice of pre-treatment outcome lags, rather than on the inclusion of covariates, because it is possible to systematically analyze the inclusion of pre-intervention outcomes lags in a way that encompasses all applications, while covariates may differ in complex ways from one application to another. We consider the possibility of specification searching in the decision to include covariates in our empirical applications. 6 Appendices are available at the end of this article as it appears in JPAM online. Go to the publisher's website and use the search engine to locate the article at http://onlinelibrary.wiley.com. 7 In Appendix B, we also consider specifications that use time-invariant covariates as predictors, in addition to functions of the pre-treatment outcomes. All results remain similar. This package solves the nested minimization problem described by equations (1) and (2). We specify the optimization method to be BFGS only and use optimization routine Low Rank Quadratic Programming when Interior Point optimization routine does not converge. 24 When we adopt this decision rule in our MC simulations, then the probability of rejecting the null at 5 percent for all specifications is lower than 1 percent in all scenarios. If we discard specifications 6 and 7, then this rejection rate ranges from 1 percent when T 0 = 12 to 2.8 percent when T 0 = 400. 25 Following the best practices in terms of transparency and replicability, Hainmueller (2014) made their dataset and replication ...

show abstract

Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets

Cited by 159 publications

References 67 publications

The Pot Rush: Is Legalized Marijuana a Positive Local Amenity?

The Pot Rush: Is Legalized Marijuana a Positive Local Amenity?

An exact and robust conformal inference method for counterfactual and synthetic controls

Cherry Picking with Synthetic Controls

Contact Info

Product

Resources

About