Convergence rates of vector-valued local polynomial regression

Abstract

Non-parametric estimation of functions as well as their derivatives by means of localpolynomial regression is a subject that was studied in the literature since the late 1970's. Given a set of noisy samples of a C k smooth function, we perform a local polynomial fit, and by taking its m-th derivative we obtain an estimate for the m-th function derivative. The known optimal rates of convergence for this problem for a k-times smooth function f : R d → R are n − k−m 2k+d . However in modern applications it is often the case that we have to estimate a function operating to R D , for D ≫ d extremely large. In this work, we prove that these same rates of convergence are also achievable by local-polynomial regression in case of a high dimensional target, given some assumptions on the noise distribution. This result is an extension to Stone's seminal work from 1980 to the regime of high-dimensional target domain. In addition, we unveil a connection between the failure probability ε and the number of samples required to achieve the optimal rates.