On the Linear Convergence of the ADMM in Decentralized Consensus Optimization

Shi, Wei; Ling, Qing; Yuan, Kun; Wu, Gang; Yin, Wotao

doi:10.1109/tsp.2014.2304432

Cited by 782 publications

(668 citation statements)

References 32 publications

Supporting

Mentioning

646

Contrasting

Order By: Relevance

“…This derivation will help clarify the origin of the O(µ 2 max ) bias from (8) in the standard diffusion implementation.…”

Section: Development Of Exact Diffusionmentioning

confidence: 94%

“…However, since V is rank-deficient, there can be multiple solutions Y satisfying (35). Using an argument similar to [8], [9], we can show that among all possible Y , there is a unique solution Y o lying in the column span of V. Using the above two lemmas, we can show that (W i , Y i ) generated through the exact diffusion recursion (29) will converge exponentially fast to (W , Y o ). We first introduce a common assumption.…”

Section: Lemma 1 (Optimality Condition)mentioning

confidence: 95%

“…Emails:{kunyuan,ybc,xiaochuanzhao, sayed}@ucla.edu alternating direction method of multipliers (ADMM) [6] and its variants [7]. It is shown in [8] that distributed ADMM with constant penalty coefficients can converge exponentially fast to the exact global solution w o . However, distributed ADMM solutions are computationally expensive since they necessitate the solution of optimal sub-problems at each iteration.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exact diffusion strategy for optimization by networked agents

Yuan

Ying

Zhao

et al. 2017

2017 25th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

Abstract-This work develops a distributed optimization algorithm with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown to have a wider stability range and superior convergence performance than the EXTRA consensus strategy. The exact diffusion solution is also applicable to non-symmetric left-stochastic combination matrices, while most earlier developments on exact consensus implementations are limited to doubly-stochastic matrices or right-stochastic matrices; these latter policies impose stringent constraints on the network topology. Stability and convergence results are noted, along with numerical simulations to illustrate the conclusions.

show abstract

“…This derivation will help clarify the origin of the O(µ 2 max ) bias from (8) in the standard diffusion implementation.…”

Section: Development Of Exact Diffusionmentioning

confidence: 94%

Section: Lemma 1 (Optimality Condition)mentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exact diffusion strategy for optimization by networked agents

Yuan

Ying

Zhao

et al. 2017

2017 25th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

show abstract

“…For a stationary network with bi-directional communication, the existing algorithms include the primal-dual domain methods such as the decentralized alternating direction method of multipliers (DADMM) [13,14], and the primal domain methods including the distributed subgradient method (DSM) [9]. Both algorithms do not take advantage of the smooth+nonsmooth structure.…”

Section: Introductionmentioning

confidence: 99%

A fast proximal gradient algorithm for decentralized composite optimization over directed networks

Zeng

Wang

2017

Systems & Control Letters

View full text Add to dashboard Cite

This paper proposes a fast decentralized algorithm for solving a consensus optimization problem defined in a directed networked multi-agent system, where the local objective functions have the smooth+nonsmooth composite form, and are possibly nonconvex. Examples of such problems include decentralized compressed sensing and constrained quadratic programming problems, as well as many decentralized regularization problems. We extend the existing algorithms PG-EXTRA and ExtraPush to a new algorithm PG-ExtraPush for composite consensus optimization over a directed network. This algorithm takes advantage of the proximity operator like in PG-EXTRA to deal with the nonsmooth term, and employs the push-sum protocol like in ExtraPush to tackle the bias introduced by the directed network. With a proper step size, we show that PGExtraPush converges to an optimal solution at a linear rate 1 under some regular assumptions.We conduct a series of numerical experiments to show the effectiveness of the proposed algorithm. Specifically, with a proper step size, PG-ExtraPush performs linear rates in most of cases, even in some nonconvex cases, and is significantly faster than Subgradient-Push, even if the latter uses a hand-optimized step size. The established theoretical results are also verified by the numerical results.Keywords: Decentralized optimization; directed network; composite objective; nonconvex; consensus.✩ The work of J. Zeng is supported in part by the National Natural Science Foundation of China (Grants No. 61603162, 11401462).1 In this paper, we use the notion of R-linear rate, i.e., a sequence {x t } converging to x * at an R-linear rate means that x t − x * ≤ Cρ t for some constants C > 0 and ρ ∈ (0, 1).

show abstract

“…Decentralized consensus optimization problems are an important class of these problems [2]. To solve these problems, distributed methods-which only require the agents to locally exchange information between each other-gain a growing interest with every passing day.…”

Section: Introductionmentioning

confidence: 99%

Distributed Newton Methods for Strictly Convex Consensus Optimization Problems in Multi-Agent Networks

2017

View full text Add to dashboard Cite

Abstract:Various distributed optimization methods have been developed for consensus optimization problems in multi-agent networks. Most of these methods only use gradient or subgradient information of the objective functions, which suffer from slow convergence rate. Recently, a distributed Newton method whose appeal stems from the use of second-order information and its fast convergence rate has been devised for the network utility maximization (NUM) problem. This paper contributes to this method by adjusting it to a special kind of consensus optimization problem in two different multi-agent networks. For networks with Hamilton path, the distributed Newton method is modified by exploiting a novel matrix splitting techniques. For general connected multi-agent networks, the algorithm is trimmed by combining the matrix splitting technique and the spanning tree for this consensus optimization problems. The convergence analyses show that both modified distributed Newton methods enable the nodes across the network to achieve a global optimal solution in a distributed manner. Finally, the distributed Newton method is applied to solve a problem which is motivated by the Kuramoto model of coupled nonlinear oscillators and the numerical results illustrate the performance of the proposed algorithm.

show abstract

On the Linear Convergence of the ADMM in Decentralized Consensus Optimization

Cited by 782 publications

References 32 publications

Exact diffusion strategy for optimization by networked agents

Exact diffusion strategy for optimization by networked agents

A fast proximal gradient algorithm for decentralized composite optimization over directed networks

Distributed Newton Methods for Strictly Convex Consensus Optimization Problems in Multi-Agent Networks

Contact Info

Product

Resources

About