Distributed dual averaging for convex optimization under communication delays

Abstract

In this paper we extend and analyze the distributed dual averaging algorithm [1] to handle communication delays and general stochastic consensus protocols. Assuming each network link experiences some fixed bounded delay, we show that distributed dual averaging converges and the error decays at a rate O(T −0.5 ) where T is the number of iterations. This bound is an improvement over [1] by a logarithmic factor in T for networks of fixed size. Finally, we extend the algorithm to the case of using general non-averaging consensus protocols. We prove that the bias introduced in the optimization can be removed by a simple correction that depends on the stationary distribution of the consensus matrix. I. INTRODUCTIONIn this paper we extend and analyze the distributed dual averaging algorithm [1]. We employ the fixed delay model introduced in [2] and show that distributed dual averaging still converges in the presence of finite and fixed communication delays. In addition, using a different bounding technique than [1], for a fixed network size, we improve on the convergence rate in terms of number of iterations by removing a logarithmic factor. Finally, we analyze the case where a general (non-averaging) consensus protocol is used. We explain and illustrate in simulation how the use of nondoubly stochastic consensus matrices biases the optimization. The issue is not however essential and we prove that a simple correction removes the bias.Over the last few years, the dramatic increase in available data has made imperative the use of parallel and distributed algorithms for solving large scale optimization and machine learning problems (see for example [3], [4]). Among the numerous possible choices, fully distributed algorithms combining some version of local optimization with a distributed consensus protocol are an appealing option [1], [4]- [7]. With such an approach, all computing nodes have the same role in the optimization procedure, thereby eliminating single points of failure and increasing robustness. This is important in large scale systems where machines may fail during the computation. This approach also has increased flexibility at adding more computational resources. At the same time, these algorithms are simple to implement and avoid the bookkeeping needed for more intricate hierarchical algorithms.The main focus of this paper is the analysis and extension of the distributed dual averaging algorithm. For practical application, it is important to know how the algorithm behaves

Push-Sum Distributed Dual Averaging for convex optimization

“…In [8] it is proven that restricting to doubly stochastic consensus protocols in distributed dual averaging is not necessary and it is still possible to converge to the optimum with a general row stochastic protocol P . However there are multiple reasons why using a row stochastic matrix may not be desirable.…”

“…However there are multiple reasons why using a row stochastic matrix may not be desirable. The bias correction described in [8] requires knowledge of the stationary distribution of P in advance which is restrictive. Moreover, with a time-varying consensus protocol P (t), we may not even be able to specify the stationary distribution beyond its expectation and variance [9] or may only be able to achieve average consensus in expectation [10].…”

“…The result holds for one directional communication using arbitrary column stochastic consensus protocols P that respect the structure of the strongly connected network. In contrast to [8], the bias in the optimization introduced by P is corrected without requiring knowledge of the stationary distribution of P or the network size. In the last part of the paper, we discuss how our convergence result can be extended to accommodate communication delays and a fully asynchronous version of the algorithm.…”

“…For example, not all (directed) networks admit a doubly stochastic matrix [6]. However, relinquishing double stochasticity can introduce bias in the optimization [7], [8]. Moreover, it is desirable to only rely on one directional communication between nodes because in a bi-directional case where each node blocks until it receives a response, deadlocks can occur when the network has cycles.…”

Distributed Nesterov-like gradient algorithms

“…(Here by private cost f i (·) we mean that the function f i (·) is known only by node i.) Existing distributed (sub)gradient algorithms, e.g., the algorithm proposed in [2] and extended and analyzed in [3], [2], [4], [5], [6], [7], and the one proposed in [8] and extended and analyzed in [9], [10], converge slowly. For example, assuming possibly non-differentiable, convex f i 's, with bounded gradients over the constraint set, algorithm [2] with constant step size ↵, after k iterations, has the error in the cost O(↵+1/(↵k)), which, for the optimized ↵, gives O(1/✏ 2 ) convergence time.…”