Convergence of multi-block Bregman ADMM for nonconvex composite problems

Abstract

The alternating direction method with multipliers (ADMM) has been one of most powerful and successful methods for solving various composite problems. The convergence of the conventional ADMM (i.e., 2-block) for convex objective functions has been justified for a long time, and its convergence for nonconvex objective functions has, however, been established very recently. The multi-block ADMM, a natural extension of ADMM, is a widely used scheme and has also been found very useful in solving various nonconvex optimization problems. It is thus expected to establish convergence theory of the multi-block ADMM under nonconvex frameworks. In this paper we present a Bregman modification of 3-block ADMM and establish its convergence for a large family of nonconvex functions. We further extend the convergence results to the N-block case (N ≥ 3), which underlines the feasibility of multi-block ADMM applications in nonconvex settings. Finally, we present a simulation study and a real-world application to support the correctness of the obtained theoretical assertions.

Cauchy Noise Removal by Nonconvex ADMM with Convergence Guarantees

“…Referring to [47,51], the function F (u) is semi-algebraic, and G(v) is a real analytic function. Thus, we conclude that L β satisfies the KL inequality [8].…”

“…Recently, researchers have discovered some useful convergence properties of the optimization algorithms for solving nonconvex minimization problems [24,47,48,53]. In particular, the paper [48] established the global convergence (to a stationary point) of the alternating direction method of multipliers (ADMM) for nonconvex nonsmooth optimization with linear constraints.…”

Dynamic Behavior Analysis via Structured Rank Minimization

“…For the multiblock separable convex problems, with three or more blocks of variables, it is known that the original ADMM is not necessarily convergent (Chen et al 2016). On the other hand, theoretical convergence analysis of the ADMM for nonconvex problems is rather limited, making either assumptions on the iterates of the algorithm (Xu et al 2012;Magnusson et al 2016) or dealing with special non-convex models (Li and Pong 2015;Wang et al 2014aWang et al , 2015, none of which is applicable for the proposed optimization problem (12). However, it is worth noting that the ADMM exhibits good numerical performance in non-convex problems such as nonnegative matrix factorization (Sun and Févotte 2014), tensor decomposition (Liavas and Sidiropoulos 2015), matrix separation (Shen et al 2014;Papamakarios et al 2014), matrix completion (Xu et al 2012), motion segmentation , to mention but a few.…”

$\ell_0$ -Motivated Low-Rank Sparse Subspace Clustering

“…To solve corresponding optimization problems we derive algorithms based on computationally efficient Alternating Direction Method of Multipliers (ADMM) [65]. Although ADMM has been successfully applied for many nonconvex problems [66]- [68], only recent theoretical results establish convergence of ADMM for certain nonconvex functions [69]- [72]. For GMC regularization, we show that the sequence generated by the algorithm is bounded and prove that any limit point of the iteration sequence is a stationary point.…”