GTAdam: Gradient Tracking With Adaptive Momentum for Distributed Online Optimization

Abstract

This paper deals with a network of computing agents aiming to solve an online optimization problem in a distributed fashion, i.e., by means of local computation and communication, without any central coordinator. We propose the gradient tracking with adaptive momentum estimation (GTAdam) distributed algorithm, which combines a gradient tracking mechanism with first and second order momentum estimates of the gradient. The algorithm is analyzed in the online setting for strongly convex cost functions with Lipschitz continuous gradients. We provide an upper bound for the dynamic regret given by a term related to the initial conditions and another term related to the temporal variations of the objective functions. Moreover, a linear convergence rate is guaranteed in the static setup. The algorithm is tested on a time-varying classification problem, on a (moving) target localization problem, and in a stochastic optimization setup from image classification. In these numerical experiments from multi-agent learning, GTAdam outperforms state-of-the-art distributed optimization methods.

Distributed Stochastic Gradient Tracking Algorithm With Variance Reduction for Non-Convex Optimization

“…A distributed adaptive gradient algorithm with bounded stepsizes was further studied in [31] to improve the generalization capacity. By introducing a GT estimator, a novel distributed adaptive algorithm was developed in the notable work [38], which is proved to achieve a linear convergence rate under the strongly-convex setting.…”

“…The adaptive stepsizes are generated according to the historical gradients, enabling the algorithm to automatically coordinate the stepsizes among dimensions when the gradients are sparse. Inspired by [38], we utilize a GT estimator to aggregate the gradients over the network. Moreover, an clipping operator is used to mitigate the negative effects of extreme adaptive stepsizes.…”

“…Compared to the GT-based distributed adaptive gradient algorithm for a strongly-convex problem in [38], our work aims to investigate the stationarity performance of the proposed Algorithm 1 for a more challenging distributed stochastic non-convex optimization problem. A notable characteristic of Algorithm 1 is the utilization of a clipping operator as in (6), which distinguishes our algorithm from the algorithm in [38] with an additive bias term. Specifically, in [38], an easily implemented additive bias term ϵ is employed on each v t,i to ensure that the adaptive vector v t,i + ϵ is consistently greater than or equal to the constant ϵ.…”

“…A notable characteristic of Algorithm 1 is the utilization of a clipping operator as in (6), which distinguishes our algorithm from the algorithm in [38] with an additive bias term. Specifically, in [38], an easily implemented additive bias term ϵ is employed on each v t,i to ensure that the adaptive vector v t,i + ϵ is consistently greater than or equal to the constant ϵ. In contrast, our clipping operator provides a more direct approach to mitigate the negative impact of extreme stepsizes, since the adaptive vector will be clipped only if v t,i is smaller or larger than the threshold v min or v max , respectively.…”

“…Furthermore, we provide a rigorous stationarity analysis for our Algorithm 1, which establishes an explicit upper bound on the optimality gap, as demonstrated in Corollary 1 below. It is important to note that our analysis approach can also be extended and applied to the algorithm in [38] by adjusting the coefficients v min , v max in our analysis according to the value of additive bias term ϵ in [38].…”