Push–Pull Gradient Methods for Distributed Optimization in Networks

Abstract

In this paper, we focus on solving a distributed convex optimization problem in a network, where each agent has its own convex cost function and the goal is to minimize the sum of the agents' cost functions while obeying the network connectivity structure. In order to minimize the sum of the cost functions, we consider new distributed gradient-based methods where each node maintains two estimates, namely, an estimate of the optimal decision variable and an estimate of the gradient for the average of the agents' objective functions. From the viewpoint of an agent, the information about the gradients is pushed to the neighbors, while the information about the decision variable is pulled from the neighbors hence giving the name "push-pull gradient methods". This name is also due to the consideration of the implementation aspect: the push-communication-protocol and the pull-communication-protocol are respectively employed to implement certain steps in the numerical schemes. The methods utilize two different graphs for the information exchange among agents, and as such, unify the algorithms with different types of distributed architecture, including decentralized (peer-to-peer), centralized (master-slave), and semi-centralized (leader-follower) architecture. We show that the proposed algorithms and their many variants converge linearly for strongly convex and smooth objective functions over a network (possibly with unidirectional data links) in both synchronous and asynchronous random-gossip settings. We numerically evaluate our proposed algorithm for both static and time-varying graphs, and find that the algorithms are competitive as compared to other linearly convergent schemes. ). 1 applications in learning can be found. Parallel, coordinated, and asynchronous algorithms were discussed in [20] and the references therein. The reader is also referred to the recent paper [15] and the references therein for a comprehensive survey on distributed optimization algorithms.In the first part of this paper, we introduce a novel gradient-based algorithm (Push-Pull) for distributed (consensus-based) optimization in directed graphs. Unlike the push-sum type protocol used in the previous literature [16,36], our algorithm uses a row stochastic matrix for the mixing of the decision variables, while it employs a column stochastic matrix for tracking the average gradients. Although motivated by a fully decentralized scheme, we show that Push-Pull can work both in fully decentralized networks and in two-tier networks.Gossip-based communication protocols are popular choices for distributed computation due to their low communication costs [1,10,8,11]. In the second part of this paper, we consider a random-gossip push-pull algorithm (G-Push-Pull) where at each iteration, an agent wakes up uniformly randomly and communicates with one or two of its neighbors. Both Push-Pull and G-Push-Pull have different variants. We show that they all converge linearly to the optimal solution for strongly convex and smooth objective functions.

Distributed stochastic gradient tracking methods

“…For another example, in simulation-based optimization, the gradient estimation often incurs noise that can be due to various sources, such as modeling and discretization errors, incomplete convergence, and finite sample size for Monte-Carlo methods [22]. Distributed algorithms dealing with problem (1) have been studied extensively in the literature [56,36,37,28,19,20,52,13,46,34,45]. Recently, there has been considerable interest in distributed implementation of stochastic gradient algorithms [48,54,14,3,5,55,6,9,10,7,32,24,26,40,51,41,18].…”

Distributed Nesterov Gradient Methods Over Arbitrary Graphs

“…It is shown in [19] that the oracle complexity with doublystochastic weights is O(Q 2 log 1 ). Extensions of AB include: non-coordinated step-sizes and heavy-ball momentum [32]; time-varying graphs [36], [37]; analysis for non-convex functions [38]. Related work on distributed Nesterov-type methods can be found in [39]- [41], which is restricted to undirected graphs.…”

“…Since these variants only require CS weights, AB and ABN are preferable due to their faster convergence. It is further straightforward to conceive a timevarying implementation of ABN and FROZEN over gossip based protocols or random graphs, see e.g., the related work in [36], [37] on non-accelerated methods. Asynchronous schemes may also be derived following the methodologies studied in [42], [43].…”

Optimales Energie-Management über verteilte, beschränkte Gradientenverfahren

“…In [8] und [12] werden verteilte Gradientenverfahren für unbeschränkte Optimierungen betrachtet. [8] verwendet das Push-Sum-Konsensus-Protokoll zur Berechnung von verteilten Informationen und beweist, dass eine Konvergenz auf ein Optimum auch bei zeitlichen Änderungen der Kommunikationsverbindungen möglich ist.…”

“…[8] verwendet das Push-Sum-Konsensus-Protokoll zur Berechnung von verteilten Informationen und beweist, dass eine Konvergenz auf ein Optimum auch bei zeitlichen Änderungen der Kommunikationsverbindungen möglich ist. [12] untersucht ein Push-Pull-Konsensus-Protokoll zur Lösung eines verteilten Optimierungsproblems und kann eine lineare Konvergenz des dort vorgestellten Algorithmus nachweisen. [5] untersucht ebenfalls unbeschränkte Gradi-entenverfahren, die auf einer verteilten Ausführung des Nesterov-Gradienten-Verfahrens beruhen und dadurch eine schnelle Konvergenzrate aufweisen.…”