Two-Step Estimation and Inference with Possibly Many Included Covariates

Cattaneo, Matias D.; Jansson, Michael; Ma, Xinwei

doi:10.1093/restud/rdy053

Cited by 53 publications

(42 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, cross‐fitting paired with local robustness may yield weaker smoothness conditions by providing “underfitting” robustness, that is, weakening bias‐related assumptions (Chernozhukov et al (2018)), but the cost may be too high here. Weaker variance‐related assumptions, or “overfitting” robustness (Cattaneo, Jansson, and Ma (2019)), may also be possible following deep learning, but are less automatic at present. Other methods for causal inference under relaxed assumptions may be useful here, such as extensions to doubly robust inference (Tan (2020)) or robust inverse weighting (Ma and Wang (2018)).…”

Section: Inference After Deep Learningmentioning

confidence: 99%

Deep Neural Networks for Estimation and Inference

Farrell

Liang

Misra

2021

ECTA

236

144

View full text Add to dashboard Cite

We study deep neural networks and their use in semiparametric inference. We establish novel nonasymptotic high probability bounds for deep feedforward neural nets. These deliver rates of convergence that are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second‐step inference after first‐step estimation with deep learning, a result also new to the literature. Our nonasymptotic high probability bounds, and the subsequent semiparametric inference, treat the current standard architecture: fully connected feedforward neural networks (multilayer perceptrons), with the now‐common rectified linear unit activation function, unbounded weights, and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed‐width, very deep networks. We establish the nonasymptotic bounds for these deep nets for a general class of nonparametric regression‐type loss functions, which includes as special cases least squares, logistic regression, and other generalized linear models. We then apply our theory to develop semiparametric inference, focusing on causal parameters for concreteness, and demonstrate the effectiveness of deep learning with an empirical application to direct mail marketing.

show abstract

Section: Inference After Deep Learningmentioning

confidence: 99%

Deep Neural Networks for Estimation and Inference

Farrell

Liang

Misra

2021

ECTA

236

144

View full text Add to dashboard Cite

show abstract

“…Chernozhukov et al (2018) and Newey and Robins (2018) utilized related cross-fitting techniques to reduce bias in high-dimensional estimation problems that feature machine-learning estimators. Cattaneo, Jansson, and Ma (2019) characterized the bias in (nonlinear) two-step estimators when the first step features a high-dimensional linear regression.…”

Section: Resultsmentioning

confidence: 99%

Heteroscedasticity-Robust Inference in Linear Regression Models With Many Covariates

Jochmans

2020

Journal of the American Statistical Association

View full text Add to dashboard Cite

We consider inference in linear regression models that is robust to heteroscedasticity and the presence of many control variables. When the number of control variables increases at the same rate as the sample size the usual heteroscedasticity-robust estimators of the covariance matrix are inconsistent. Hence, tests based on these estimators are size distorted even in large samples. An alternative covariance-matrix estimator for such a setting is presented that complements recent work by Cattaneo, Jansson, and Newey. We provide high-level conditions for our approach to deliver (asymptotically) size-correct inference as well as more primitive conditions for three special cases. Simulation results and an empirical illustration to inference on the union premium are also provided. Supplementary materials for this article are available online.

show abstract

“…Cattaneo et al . () investigate the consequences of including a (moderately) large number of regressors in the first‐step regression on parameter estimates in the second step, in a general GMM framework with generated regressors. The critical rate of numerosity of regressors turns out to be

m = O (\sqrt{n})

, in which case the numerosity of first‐step regressors induces an inconsistency bias in the distribution of the second‐step estimates, in addition to higher order effects on the asymptotic variance.…”

Section: Many Regressorsmentioning

confidence: 99%

Many Instruments and/or Regressors: A Friendly Guide

Anatolyev

2018

Journal of Economic Surveys

View full text Add to dashboard Cite

This paper surveys the state of the art in the econometrics of regression models with many instruments or many regressors based on alternative -namely, dimension -asymptotics. We list critical results of dimension asymptotics that lead to better approximations of properties of familiar and alternative estimators and tests when the instruments and/or regressors are numerous. Then, we consider the problem of estimation and inference in the basic linear instrumental variables regression setup with many strong instruments. We describe the failures of conventional estimation and inference, as well as alternative tools that restore consistency and validity. We then add various other features to the basic model such as heteroskedasticity, instrument weakness, etc., in each case providing a review of the existing tools for proper estimation and inference. Subsequently, we consider a related but different problem of estimation and testing in a linear mean regression with many regressors. We also describe various extensions and connections to other settings, such as panel data models, spatial models, time series models, and so on. Finally, we provide practical guidance regarding which tools are most suitable to use in various situations when many instruments and/or regressors turn out to be an issue.Most recent theoretical literature tends to deal with all of the possible complications involved, and thus the degree of sophistication can become a stumbling block for applied researchers. The survey is structured so that complications are gradually scaffolded on an easily comprehensible skeleton model -a linear homoskedastic IVs model. In this section, we describe a road map for how the survey is organized.Section 3 presents the statistical foundations for many instrument/regressor asymptotics; an uninterested reader may skip this section during a first reading and jump to Section 4. First, we reiterate the idea of alternative asymptotic approximations, in particular, dimension asymptotics. Second, the historical evolution of limit theorems for bilinear forms -the main building block of many instrument/regressor MANY INSTRUMENTS AND/OR REGRESSORS: A FRIENDLY GUIDE 691 asymptotics -is shown, starting with an example of a simplest quadratic form, to show the differences between the approaches -the traditional asymptotics and dimension asymptotics -and how they interlock. Third, a certain important asymptotic property of the diagonal of the projection matrix is discussed.In Section 4, a linear homoskedastic IVs model is considered in detail. The reasons why 2SLS estimation fails and various remedies for such failures (bias correction, jackknifing, and LIML) are discussed. The alternative asymptotic distributions of consistent estimators are presented, initially for the case of normal errors. Then, we relax error normality to examine the arising complications and discuss how they can be handled. In addition to inferential procedures regarding parameter restrictions, the issues of asymptotic efficiency and specification testing ar...

show abstract

Two-Step Estimation and Inference with Possibly Many Included Covariates

Cited by 53 publications

References 54 publications

Deep Neural Networks for Estimation and Inference

Deep Neural Networks for Estimation and Inference

Heteroscedasticity-Robust Inference in Linear Regression Models With Many Covariates

Many Instruments and/or Regressors: A Friendly Guide

Contact Info

Product

Resources

About