Rebar: Reinforcing a Matching Estimator With Predictions From High-Dimensional Covariates

Abstract

In causal matching designs, some control subjects are often left unmatched, and some covariates are often left unmodeled. This article introduces “rebar,” a method using high-dimensional modeling to incorporate these commonly discarded data without sacrificing the integrity of the matching design. After constructing a match, a researcher uses the unmatched control subjects—the remnant—to fit a machine learning model predicting control potential outcomes as a function of the full covariate matrix. The resulting predictions in the matched set are used to adjust the causal estimate to reduce confounding bias. We present theoretical results to justify the method’s bias-reducing properties as well as a simulation study that demonstrates them. Additionally, we illustrate the method in an evaluation of a school-level comprehensive educational reform program in Arizona.

Mathematics Curriculum Effects on Student Achievement in California

“…Thus, these schools are outside of our evaluation sample. Following Sales, Hansen, and Rowan (2014), we start by estimating the following linear regression model using the remnant data:…”

“…Thus, these schools are outside of our evaluation sample. Following Sales, Hansen, and Rowan (2014), we start by estimating the following linear regression model using the remnant data: In Equation (C1), Y sdt is a Grade 3 math test score for school s in district d in year t , and X s and X d are defined as above. 22 After estimating Equation (C1), we store the coefficient estimates true α ^ 1 t and true α ^ 2 t and construct the following residualized test score outcome for each school in our analytic sample in each year: In Equation (C2), Y sdt is the Grade 3 test score for school s in district d in year t for a school that adopted one of the four primary curricula.…”

“…This procedure is very similar to restricted ordinary least squares (OLS), with the added benefit that the adjustment parameters true α ^ 1 t and true α ^ 2 t are estimated entirely out of sample. Using an out-of-sample “training set” for the outcome model has several conceptual benefits over using in-sample data (as was the case with OLS) as described by Sales et al (2014). In our application, it addresses the concern that bias could be introduced by the OLS models if the covariate coefficients are disproportionately influenced by schools in the control condition, which dominate our sample.…”

“…It blends aspects of the restricted-OLS and matching strategies. The fundamental idea, taken from Sales, Hansen, and Rowan (2014), is to pull in data from outside of the evaluation-that is, "remnant data"-to regression-adjust outcomes prior to matching. Sales et al suggest several potential uses of remnant-based residualization; in our application, the appeal is that the procedure can remove noise from the outcome data before matching occurs, thereby improving the precision of our estimates.…”

Limitless Regression Discontinuity

“…In this analysis, true Y ^ C ( R i ) cannot itself be influenced by Maria since it is based on a model fit to pre-Maria death counts (cf. Sales, Hansen, & Rowan, 2018). Hence, the effect of Maria on E (i.e., E T − E C ) is exactly equal to its effect on Y .…”