Estimating treatment effect heterogeneity in randomized program evaluation

Abstract

When evaluating the efficacy of social programs and medical treatments using randomized experiments, the estimated overall average causal effect alone is often of limited value and the researchers must investigate when the treatments do and do not work. Indeed, the estimation of treatment effect heterogeneity plays an essential role in (1) selecting the most effective treatment from a large number of available treatments, (2) ascertaining subpopulations for which a treatment is effective or harmful, (3) designing individualized optimal treatment regimes, (4) testing for the existence or lack of heterogeneous treatment effects, and (5) generalizing causal effect estimates obtained from an experimental sample to a target population. In this paper, we formulate the estimation of heterogeneous treatment effects as a variable selection problem. We propose a method that adapts the Support Vector Machine classifier by placing separate sparsity constraints over the pre-treatment parameters and causal heterogeneity parameters of interest. The proposed method is motivated by and applied to two well-known randomized evaluation studies in the social sciences. Our method selects the most effective voter mobilization strategies from a large number of alternative strategies, and it also identifies the characteristics of workers who greatly benefit from (or are negatively affected by) a job training program. In our simulation studies, we find that the proposed method often outperforms some commonly used alternatives. . The proposed methods can be implemented via open-source software FindIt [Ratkovic and Imai (2012)], which is freely available at the Comprehensive R Archive Network (http://cran.r-project.org/package=FindIt). This software also contains the results of our empirical analysis.

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

“…Imai and Ratkovic [2013], Signorovitch [2007], Tian et al [2014] and Weisberg and Pontes [2015] develop lasso-like methods for causal inference in a sparse high-dimensional linear setting. Beygelzimer and Langford [2009], Dudík et al [2011], and others discuss procedures for transforming outcomes that enable off-the-shelf loss minimization methods to be used for optimal treatment policy estimation.…”

The State of Applied Econometrics: Causality and Policy Evaluation

“…Imai and Ratkovic (2013) estimate a LASSO regression model with the treatment indicator interacted with covariates, and uses LASSO as a variable selection algorithm for determining which covariates are most important. In using this approach, it may be prudent to perform some supplementary analysis to verify that the method is not overfitting; for example, one could use a sample-splitting approach, using half of the data to estimate the LASSO regression and then comparing the results to an ordinary least squares regression with the variables selected by LASSO in the other half of the data.…”

Personalised treatment assignment maximising expected benefit with smooth hinge loss

“…The reason why we compare our sHinge method with the ROWSi is that Xu et al (2015) showed by simulation that the ROWSi was superior over the method solving (9) with h k replaced by the hinge loss, which was proposed in Zhao et al (2012) except that LASSO penalty instead of L 2 penalty was used for variable selection. Xu et al (2015) also showed by simulation that ROWSi was superior over other four recently proposed methods, the interaction tree by Su, Tsai, Wang, Nickerson, and Li (2009), the virtual twins by Foster, Taylor, and Ruberg (2011), the logistic regression with LASSO penalty by Qian and Murphy (2011), and the FindIt by Imai and Ratkovic (2013).…”