Optimal adaptive inference in random design binary regression

Abstract

We construct confidence sets for the regression function in nonparametric binary regression with an unknown design density-a nuisance parameter in the problem. These confidence sets are adaptive in L 2 loss over a continuous class of Sobolev type spaces. Adaptation holds in the smoothness of the regression function, over the maximal parameter spaces where adaptation is possible, provided the design density is smooth enough. We identify two key regimes -one where adaptation is possible, and one where some critical regions must be removed. We address related questions about goodness of fit testing and adaptive estimation of relevant infinite dimensional parameters.In many epidemiological studies, a binary response variable Y is independently observed on a population of individuals along with multiple covariates X to explain the variability in the response. In the context of epidemiological studies, the probability of observing a specific outcome conditional on the covariates is often referred to as the propensity score. Estimating propensity score type functions from observed data is often of interest, and these estimates are subsequently used in multiple inferential procedures such as propensity score matching (Rosenbaum and Rubin, 1983), inverse probability weighted inference (Robins, Rotnitzky and Zhao, 1994) etc. In the context of semiparametric inference for missing data type problems, a nice exposition to the importance of understanding questions of similar flavor can be found in Tsiatis (2007).Historically, regression models with binary outcomes have been approached through both parametric (McCullagh and Nelder, 1989) and nonparametric lenses (Antoniadis and Leblanc, 2000;Signorini and Jones, 2004). Although parametric regression has the natural advantage of being simpler in interpretation and implementation, it often lacks the desired complexity required to capture varieties of dependence between covariates and outcomes. Nonparametric binary regression attempts to address this question, but it has its own share of shortcomings-the two major concerns being dependence on a priori knowledge about the true underlying regression function class and ease of implementation. Motivated by these, in this paper we study inference (estimation, testing, and confidence sets) in binary regression problems under nonparametric models having random covariates with unknown design density, with primary focus on adaptation over function classes.To fix ideas, suppose we observe data (x i , y i ) n i=1 , where x i ∈ [0, 1] d and y i ∈ {0, 1}. Consider the binary regression model E(y|x) = P (y = 1|x) = f (x), y ∈ {0, 1}, x ∼ g.(0.1)AMS 2000 subject classifications: Primary 62G10, 62G20, 62C20

On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning

“…However, because IF 22,k − IF 22,k j is not an exact second-order degenerate U-statistic, to use such exponential tail bounds for hypothesis testing purposes requires a more careful analysis to obtain constants that can be estimated from data. Results along this line can be found in Mukherjee et al (2016); Mukherjee and Sen (2018) but more careful analysis is needed to obtain explicit constants. (3) It is possible to refine the strategy for multiple testing adjustment considered above, in which we distribute the desired overall type-I error uniformly to every J − j actually tested hypotheses associated with the hypothesis of actual interest H 0,2,k j (δ) (4.1).…”

Distributed Nonparametric Estimation under Communication Constraints

“…i ), slightly abusing our notations. The minimax properties and adaptive estimators for binary regression in the classical non-distributed setting are studied for instance in Mukherjee and Sen (2018). Binary regression is an example of a model where the log-likelihood-ratios W (k) d are bounded.…”

Smoothness-Adaptive Contextual Bandits

“…Later on, self-similarity was also used by Giné and Nickl (2010) to construct confidence bounds over finite intervals. Aside from these two works, the self-similarity condition has also been used in other applications, including high-dimensional sparse signal estimation (Nickl and van de Geer 2013), binary regression (Mukherjee and Sen 2018), and L p -confidence sets (Nickl and Szabó 2016), to mention but a few.…”