Uncertainty Quantification for Stochastic Approximation Limits Using Chaos Expansion

Abstract

The uncertainty quantification for the limit of a Stochastic Approximation (SA) algorithm is analyzed. In our setup, this limit ζ is defined as a zero of an intractable function and is modeled as uncertain through a parameter θ. We aim at deriving the function ζ , as well as the probabilistic distribution of ζ (θ) given a probability distribution π for θ. We introduce the so-called Uncertainty Quantification for SA (UQSA) algorithm, an SA algorithm in increasing dimension for computing the basis coefficients of a chaos expansion of θ → ζ (θ) on an orthogonal basis of a suitable Hilbert space. UQSA, run with a finite number of iterations K, returns a finite set of coefficients, providing an approximation ζ K (•) of ζ (•). We establish the almost-sure and L p-convergences in the Hilbert space of the sequence of functions ζ K (•) when the number of iterations K tends to infinity. This is done under mild, tractable conditions, uncovered by the existing literature for convergence analysis of infinite dimensional SA algorithms. For a suitable choice of the Hilbert basis, the algorithm also provides an approximation of the expectation, of the variance-covariance matrix, and of higher order moments of the quantity ζ K (θ) when θ is random with distribution π. UQSA is illustrated and the role of its design parameters is discussed numerically.

Uncertainty Quantification for Gradient and Accelerated Gradient Descent Methods on Strongly Convex Functions

“…A number of asymptotic properties are also shown in [4], including convergence, normality, and almost-sure loglog rate of convergence. In [5], they again study a SA sieve algorithm. A number of conditions are weakened from previous work, and they show asymptotic convergence assuming only standard SA local separation and a local strong convex type assumption.…”

“…This method may be considered if we want to find the variance for example, which can be written as ||u * || 2 2 . To see the effects of this method in terms of complexity, we will consider the motivating example of using Jacobi polynomials in the case where the dimension of x is 1 and of θ as d as in [5], and we wish to estimate the variance.…”

“…We truncate to a fixed level m, and then perform gradient descent a large number of times in order to construct a Monte Carlo approximation to the vector u as explained earlier. As outlined in [5], in order to balance the error of the truncation and the actual approximation from the Monte Carlo, we must have the number of samples to approximate the u * i increase as ε −1+d/(2(η−1)) , where η is the order of differentiability of x * (θ). Note that for us, each Monte Carlo sample must be obtained by running gradient descent, so this is the amount of times gradient descent has to be run to convergence, and we suffer from the curse of dimensionality for large d.…”

“…Therefore we are looking for a function x * (θ) such that for π-almost all θ we have that f (x * (θ)) is a minimum of f (x(θ)) . For concrete instances of the problem, we direct the reader to [5] and [12]. Our main assumptions are that the function f is strongly convex, we have noisy gradient information and that the model is to be optimized with a first order method (i.e.…”

“…We propose and analyze an uncertainty quantification algorithm for both gradient descent and accelerated gradient descent. While such algorithms have been considered previously in a Stochastic Approximation setting (for example, [5]), they only show convergence asymptotically. In order to obtain a non-asymptotic rate, we decompose the error at iteration k into two terms,…”