Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems

Abstract

Abstract. The alternating direction method of multipliers (ADMM) is widely used to solve large-scale linearly constrained optimization problems, convex or nonconvex, in many engineering fields. However there is a general lack of theoretical understanding of the algorithm when the objective function is nonconvex. In this paper we analyze the convergence of the ADMM for solving certain nonconvex consensus and sharing problems. We show that the classical ADMM converges to the set of stationary solutions, provided that the penalty parameter in the augmented Lagrangian is chosen to be sufficiently large. For the sharing problems, we show that the ADMM is convergent regardless of the number of variable blocks. Our analysis does not impose any assumptions on the iterates generated by the algorithm, and is broadly applicable to many ADMM variants involving proximal update rules and various flexible block selection rules.

DLM: Decentralized Linearized Alternating Direction Method of Multipliers

“…Constructing this argument is not difficult if we follow analogous proofs for the centralized ADMM; see e.g., [26], [33]. With particular note, recently [34] also proves convergence of the centralized ADMM for nonconvex sharing and consensus problems. We give the convergence result in the following theorem.…”

Hybrid Random/Deterministic Parallel Algorithms for Convex and Nonconvex Big Data Optimization

“…where (a) follows from Lemma 8 and β 2 = c 1 β 1 > 0; and (b) is due to (47) and (40), with β 3 = β β 2 (δ 2 /4).…”

“…Dual ADMM-like schemes have been proposed for problems with a specific nonconvex, additively separable F , and shown to be convergent under strong conditions; see, e.g., [40]. However, for the scale and generality of problems we are interested in, they are computationally impractical.…”

Energy Budget Aware Device-to-Device Cooperation for Mobile Videos

“…In order to meet the convergence necessity of the proposed algorithm [13], the framework is time-synchronized and the devices upload/monitor keep-alive/broadcasting messages in the appointed time slots, which are defined as broadcasting time slots. In the rest time slots for D2D cooperation, every cooperative device forwards the video streams to the neighboring devices in a time-division fashion.…”

“…In this section, we transform the solution of the convex optimization into a practical distributed control scheme based on a tool called Alternative Direction Method of Multipliers (ADMM) [13]. It is different from the shortest-path algorithms in wireless sensor networks such as Flow Augmentation algorithm (FA) and Minimum Total Energy Routing algorithm (MTE) [10].…”