Combining Matching and Synthetic Control to Tradeoff Biases From Extrapolation and Interpolation

Abstract

The synthetic control method is widely used in comparative case studies to adjust for differences in pre-treatment characteristics. A major attraction of the method is that it limits extrapolation bias that can occur when untreated units with different pre-treatment characteristics are combined using a traditional adjustment, such as a linear regression. Instead, the SC estimator is susceptible to interpolation bias because it uses a convex weighted average of the untreated units to create a synthetic untreated unit with pre-treatment characteristics similar to those of the treated unit. More traditional matching estimators exhibit the opposite behavior: they limit interpolation bias at the potential expense of extrapolation bias. We propose combining the matching and synthetic control estimators through model averaging to create an estimator called MASC. We show how to use a rolling-origin cross-validation procedure to train the MASC to resolve trade-o s between interpolation and extrapolation bias. We use a series of empirically-based placebo and Monte Carlo simulations to shed light on when the SC, matching, MASC and penalized SC estimators do (and do not) perform well. Then, we use the MASC re-examine the economic costs of conflicts and find evidence of larger effects than with SC.

LightGBM outperforms other machine learning techniques in predicting graft failure after liver transplantation: Creation of a predictive model through large‐scale analysis

“…Therefore, the rolling CV method more accurately emulates the process of generating a ML score based on existing observations and subsequently applies it prospectively to a recent cohort of patients reflecting temporal variations. 2,[23][24][25] Among the assessed algorithms, LightGBM demonstrated the most commendable performance in both the rolling CV method and shuffled CV technique on 14-day, 30-day and 90-day graft failure prediction. Nonetheless, no singular algorithm exhibited superiority across all scenarios, thereby suggesting that an ensemble approach, such as stacking, holds the potential to yield an optimal model by leveraging the strengths of diverse algorithms, 26 and this was also suggested in ML prediction for cardiac transplantation.…”

“…However, the model established in the shuffled CV approach was trained on patients who underwent transplantation both before and after the patients in the data and they are mixed regardless of their transplanted date, rendering it infeasible for prospective application. Therefore, the rolling CV method more accurately emulates the process of generating a ML score based on existing observations and subsequently applies it prospectively to a recent cohort of patients reflecting temporal variations 2,23–25 . Among the assessed algorithms, LightGBM demonstrated the most commendable performance in both the rolling CV method and shuffled CV technique on 14‐day, 30‐day and 90‐day graft failure prediction.…”

The augmented synthetic control method in public health and biomedical research

“…2,32 Unfortunately, finding a synthetic control that closely fits the treated unit in the pre-treatment period can be difficult to achieve in some settings. Additionally, even if good pre-treatment fit is achieved, the SCM estimator is susceptible to so-called “interpolation bias.” 2,11,32,39 Such bias may arise because the synthetic control providing a good approximation to Y 1 t false( 0 false) for t ≤ T 0 does not necessarily imply the synthetic control will also well approximate Y 1 t false( 0 false) for t > T 0 unless additional assumptions are made about the data generating process. Hence, the need for methods that modify SCM arises.…”

“…Units that experienced the same or a similar intervention to that of the treated unit and units with highly volatile pre-treatment outcomes should be discarded from the set of potential control units. 31 , 32 Restrictions on weights assigned to the control units will prevent extrapolation, but including units with pre-treatment outcomes very different from the treated unit could lead to so-called “interpolation bias.” 2 , 11 , 32 , 39 Data should be examined to ensure that the pre-treatment outcome trends of the control units are not systematically different from that of the treated unit. Likewise, pre-treatment covariates should be compared between the treated unit and control units.…”

Confidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects

“…, y N −1,t ), which can be obtained from a constrained regression. Generalisations of the synthetic control method can be found in, for example, Amjad et al (2018), Abadie and L'Hour (2021), Arkhangelsky et al (2021), Ben-Michael et al (2021a), Ben-Michael et al (2021b), Kellogg et al (2021), and Masini and Medeiros (2021). One can see Abadie (2021) for a comprehensive review.…”