Optimal and efficient approximations of gradients of functions with non-independent variables

Abstract

Gradients of smooth functions with non-independent variables are relevant for exploring complex models and for the optimization of functions subjected to constraints. In this paper, we investigate new and simple approximations and computations of such gradients by making use of independent, central and symmetric variables. Such approximations are well-suited for applications in which the computations of the gradients are too expansive or impossible. The derived upper-bounds of the biases of our approximations do not suffer from the curse of dimensionality for any $2$-smooth function, and theoretically improve the known results. Also, our estimators of such gradients reach the optimal (mean squared error) rate of convergence (i.e., $\mathcal{O}(N^{-1})$) for the same class of functions. Numerical comparisons based on a test case and a high-dimensional PDE model show the efficiency of our approach.