Adaptive greedy algorithm for moderately large dimensions in kernel conditional density estimation

Abstract

This paper studies the estimation of the conditional density f (x, •) of Y i given X i = x, from the observation of an i.i.d. sample (X i , Y i ) ∈ R d , i ∈ {1, . . . , n}. We assume that f depends only on r unknown components with typically r d. We provide an adaptive fully-nonparametric strategy based on kernel rules to estimate f . To select the bandwidth of our kernel rule, we propose a new fast iterative algorithm inspired by the Rodeo algorithm (Wasserman and Lafferty, 2006) to detect the sparsity structure of f . More precisely, in the minimax setting, our pointwise estimator, which is adaptive to both the regularity and the sparsity, achieves the quasi-optimal rate of convergence. Our results also hold for density estimation. The computational complexity of our method is only O(dn log n). A deep numerical study shows nice performances of our approach.

Minimax optimal conditional density estimation under total variation smoothness

“…The difference in the two problem settings renders the comparison of the resulting minimax rates nonproductive. In addition, the works [20,21,22,23] use different loss functions, and different assumptions from the present paper which makes it difficult to directly compare the results.…”

“…In a more recent work, [20], for instance, studies conditional density estimation (CDE) under a Hellinger loss function and proposes an estimate which uses the data to select a function among at most countable collections of candidates. On the other hand [21,22,23] study minimax CDE under the L 2 loss. In detail, [21] propose a piecewise polynomial estimator selected from using the data over a collection of given partitions.…”

“…[22] define two adaptive procedures of estima-tion tuned by using the Goldenshluger and Lepski methodology -the first one is kernel based, while the second uses projection based methods. Finally, [23] provide an adaptive kernel based nonparametric strategy for CDE.…”