Non-asymptotic error bounds for constant stepsize stochastic approximation for tracking mobile agents

Abstract

This work revisits the constant stepsize stochastic approximation algorithm for tracking a slowly moving target and obtains a bound for the tracking error that is valid for the entire time axis, using the Alekseev non-linear variation of constants formula. It is the first non-asymptptic bound for the entire time axis in the sense that it is not based on the vanishing stepsize limit and associated limit theorems unlike prior works, and captures clearly the dependence on problem parameters and the dimension.

Convergence of Recursive Stochastic Algorithms Using Wasserstein Divergence

“…Constant stepsize RSAs often converge much faster to a neighborhood of the desired solution. This phenomenon has been observed in off-policy temporal difference learning [52], temporal difference learning with function approximation [33], tracking problems [31], and gradient descent [4], among others. Furthermore, the size of this neighborhood is usually small if the stepsize is small (so too large a stepsize may not be beneficial) [13,7].…”

On the Convergence of Consensus Algorithms with Markovian Noise and Gradient Bias

“…To the best of our knowledge, this paper provides the first finite-time convergence analysis for DSA scheme with biased updates relying on Markov samples. In addition, this work is related to the recent works on non-asymptotic analysis of SA schemes [14][15][16].…”