Linearly convergent decentralized consensus optimization with the alternating direction method of multipliers

Abstract

In the decentralized consensus optimization problem, a net work of agents minimizes the summation of their local ob jective functions on a common set of variables, allowing only information exchange among neighbors. The alternating di rection method of multipliers (ADMM) has been shown to be a powerful tool for solving the problem with empirically fast convergence. This paper establishes the linear convergence rate of the ADMM in decentralized consensus optimization. The theoretical convergence rate is a function of the network topology, properties of the local objective functions, and the algorithm parameter. This result not only gives a performance guarantee for the ADMM but also provides a guideline to ac celerate its convergence rate for the decentralized consensus optimization problems.

On the Linear Convergence of the ADMM in Decentralized Consensus Optimization

“…Besides the ADMM, existing decentralized approaches for solving (1) include belief propagation [7], incremental optimization [22], subgradient descent [15]- [17], dual averaging [18], [19], etc. Belief propagation and incremental optimization require one to predefine a tree or loop structure in the network, whereas the advantage of the ADMM, subgradient descent, and dual averaging is that they do not rely on any predefined structures.…”

Optimal Parameter Selection for the Alternating Direction Method of Multipliers (ADMM): Quadratic Problems

“…The next proposition addresses another extreme scenario when the ADMM iterates are operating on the active set of the quadratic program (18).…”

“…A few recent papers have focused on the optimal parameter selection of ADMM algorithm for some variations of distributed convex programming subject to linear equality constraints e.g. [17], [18].The paper is organized as follows. In Section II, we derive some preliminary results on fixed-point iterations and review the necessary background on the ADMM method.…”

“…A few recent papers have focused on the optimal parameter selection of ADMM algorithm for some variations of distributed convex programming subject to linear equality constraints e.g. [17], [18].…”

Linear Convergence Rate of a Class of Distributed Augmented Lagrangian Algorithms

“…Finally, references [10], [11] analyze the distributed ADMM method therein when the costs are strongly convex and have Lipschitz continuous gradients. The method in [10], [11] corresponds to our deterministic Jacobi variant when τ = 1. With respect to our results, the bounds in [10], [11] are tighter than ours for the method they study.…”