Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

Abstract

Although published works rarely include causal estimates from more than a few model specifications, authors usually choose the presented estimates from numerous trial runs readers never see. Given the often large variation in estimates across choices of control variables, functional forms, and other modeling assumptions, how can researchers ensure that the few estimates presented are accurate or representative? How do readers know that publications are not merely demonstrations that it is possible to find a specification that fits the author's favorite hypothesis? And how do we evaluate or even define statistical properties like unbiasedness or mean squared error when no unique model or estimator even exists? Matching methods, which offer the promise of causal inference with fewer assumptions, constitute one possible way forward, but crucial results in this fast-growing methodological literature are often grossly misinterpreted. We explain how to avoid these

Transitions in romantic relationships and development of self-esteem.

“…We limited the possible nearest neighbor by setting the maximum caliper width to .2 (in a subset of the matching procedures, we used a smaller caliper to reduce the imbalance between the groups). To account for differences in the sample size of the treatment and control group, we used one-to-many matching (Ming & Rosenbaum, 2000;Smith, 1997; also see Ho, Imai, King, & Stuart, 2007). 3 Fourth, we examined the matched samples with regard to their balance on all control variables.…”

Long-Term and Spillover Effects of Health Shocks on Employment and Income

“…While we cannot completely rule out such bias even when using unscheduled and urgent hospitalizations, we find no difference in the ‘pre-treatment’ income trends of hospitalized individuals and their matched controls. We estimate effects using difference-in-difference regressions with weights obtained from propensity score matching (Rosenbaum and Rubin 1983; Heckman, Ichimura, and Todd 1997; Ho et al 2007; Imbens and Wooldridge 2009). …”

“…Matching on the aforementioned indicators ensures a considerable bias reduction close to 100 percent for all variables (Ho et al 2007). While recognizing the argument that standard t-tests are heavily dependent on the sample size (Ho et al 2007; Imbens and Wooldridge 2009), it is reassuring that after matching none of the differences in average characteristics are statistically significant at any conventional level despite the huge sample size (Table A.1).…”

“…Here, we use propensity score matching as a way of “pruning” the data to obtain a comparable set of treated and control individuals. After this initial pre-processing step we run a weighted regression which renders the estimates consistent in case either the matching or the regression step, but not necessarily both, is correctly specified - the “double robustness” property (Ho et al 2007; Imbens and Wooldridge 2009). However, consistency still relies on the Conditional Independence Assumption.…”

Epidemiologic analysis of racial/ethnic disparities: A rejoinder to Frank and Exner

“…In fact, there could be an advantage to the non-parametric control achieved by matching, since it avoids reliance on a specific model form (Ho et al, 2007), but this would be consequential if there were important heterogeneity of the treatment effect by the matched covariates, a situation that Exner et al do not consider.…”