Locally robust semiparametric estimation

Chernozhukov, Victor; Escanciano, Juan Carlos; Ichimura, Hidehiko; Newey, Whitney K.; Robins, James M.

doi:10.1920/wp.cem.2016.3116

Cited by 19 publications

References 84 publications

(79 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

Simultaneous inference for Best Linear Predictor of the Conditional Average Treatment Effect and other structural functions

Chernozhukov¹,

Semenova²

2018

Self Cite

View full text Add to dashboard Cite

This paper provides estimation and inference methods for a structural function, such as Conditional Average Treatment Effect (CATE), based on modern machine learning (ML) tools. We assume that such function can be represented as a conditional expectation g(x) = E[Yη 0 |X = x] of a signal Yη 0 , where η0 is the unknown nuisance function. In addition to CATE, examples of such functions include regression function with Partially Missing Outcome and Conditional Average Partial Derivative. We approximate g(x) by a linear form p(x) β0, where p(x) is a vector of the approximating functions and β0 is the Best Linear Predictor. Plugging in the first-stage estimate η into the signal Y η , we estimate β0 via ordinary least squares of Y η on p(X). We deliver a high-quality estimate p(x) β of the pseudo-target function p(x) β0, that features (a) a pointwise Gaussian approximation of p(x0) β0 at a point x0, (b) a simultaneous Gaussian approximation of p(x) β0 uniformly over x, and (c) optimal rate of convergence of p(x) β to p(x) β0 uniformly over x. In the case the misspecification error of the linear form decays sufficiently fast, these approximations automatically hold for the target function g(x) instead of a pseudo-target p(x) β0. The first stage nuisance parameter η0 is allowed to be high-dimensional and is estimated by modern ML tools, such as neural networks, l1-shrinkage estimators, and random forest. Using our method, we estimate the average price elasticity conditional on income using Yatchew and No (2001) data and provide uniform confidence bands for the target regression function.

show abstract

Simultaneous inference for Best Linear Predictor of the Conditional Average Treatment Effect and other structural functions

Chernozhukov¹,

Semenova²

2018

Self Cite

View full text Add to dashboard Cite

show abstract

Plug-in regularized estimation of high dimensional parameters in nonlinear semiparametric models

Chernozhukov¹,

Nekipelov²,

Semenova³

et al. 2018

View full text Add to dashboard Cite

We develop a theory for estimation of a high-dimensional sparse parameter θ defined as a minimizer of a population loss function L D pθ, g 0 q which, in addition to θ, depends on a, potentially infinite dimensional, nuisance parameter g 0. Our approach is based on estimating θ via an 1-regularized minimization of a sample analog of L S pθ,ĝq, plugging in a first-stage estimateĝ, computed on a hold-out sample. We define a population loss to be (Neyman) orthogonal if the gradient of the loss with respect to θ, has pathwise derivative with respect to g equal to zero, when evaluated at the true parameter and nuisance component. We show that orthogonality implies a second-order impact of the first stage nuisance error on the second stage target parameter estimate. Our approach applies to both convex and non-convex losses, albeit the latter case requires a small adaptation of our method with a preliminary estimation step of the target parameter. Our result enables oracle convergence rates for θ under assumptions on the first stage rates, typically of the order of n´1 {4. We show how such an orthogonal loss can be constructed via a novel orthogonalization process for a general model defined by conditional moment restrictions. We apply our theory to high-dimensional versions of standard estimation problems in statistics and econometrics, such as: estimation of conditional moment models with missing data, estimation of structural utilities in games of incomplete information and estimation of treatment effects in regression models with non-linear link functions.

show abstract

Double debiased machine learning nonparametric inference with continuous treatments

Colangelo¹,

Lee

2019

View full text Add to dashboard Cite

Deep models are susceptible to learning spurious correlations, even during the post-processing. We take a closer look at the knowledge distillationa popular post-processing technique for model compression-and find that distilling with biased training data gives rise to a biased student, even when the teacher is debiased. To address this issue, we propose a simple knowledge distillation algorithm, coined DeTT (Debiasing by Teacher Transplanting). Inspired by a recent observation that the last neural net layer plays an overwhelmingly important role in debiasing, DeTT directly transplants the teacher's last layer to the student. Remaining layers are distilled by matching the feature map outputs of the student and the teacher, where the samples are reweighted to mitigate the dataset bias. Importantly, DeTT does not rely on the availability of extensive annotations on the bias-related attribute, which is typically not available during the post-processing phase. Throughout our experiments, DeTT successfully debiases the student model, consistently outperforming the baselines in terms of the worst-group accuracy.

show abstract

Locally robust semiparametric estimation

Cited by 19 publications

References 84 publications

Simultaneous inference for Best Linear Predictor of the Conditional Average Treatment Effect and other structural functions

Simultaneous inference for Best Linear Predictor of the Conditional Average Treatment Effect and other structural functions

Plug-in regularized estimation of high dimensional parameters in nonlinear semiparametric models

Double debiased machine learning nonparametric inference with continuous treatments

Contact Info

Product

Resources

About