The augmented Lagrangian method with full Jacobian decomposition and logarithmic-quadratic proximal regularization for multiple-block separable convex programming

Abstract

Abstract. We consider a separable convex minimization model whose variables are coupled by linear constraints and they are subject to the positive orthant constraints, and its objective function is in form of m functions without coupled variables. It is well recognized that when the augmented Lagrangian method (ALM) is applied to solve some concrete applications, the resulting subproblem at each iteration should be decomposed to generate solvable subproblems. When the Gauss-Seidel decomposition is implemented, this idea has inspired the alternating direction method of multiplier (for m = 2) and its variants (for m ≥ 3). When the Jacobian decomposition is considered, it has been shown that the ALM with Jacobian decomposition in its subproblem is not necessarily convergent even when m = 2 and it was suggested to regularize the decomposed subproblems with quadratic proximal terms to ensure the convergence. In this paper, we focus on the multiple-block case with m ≥ 3. We consider implementing the full Jacobian decomposition to ALM's subproblems and using the logarithmic-quadratic proximal (LQP) terms to regularize the decomposed subproblems. The resulting subproblems are all unconstrained minimization problems because the positive orthant constraints are all inactive; and they are fully eligible for parallel computation. Accordingly, the ALM with full Jacobian decomposition and LQP regularization is proposed. We also consider its inexact version which allows the subproblems to be solved inexactly. For both the exact and inexact versions, we comprehensively discuss their convergence, including their global convergence, worst-case convergence rates measured by the iteration-complexity in both the ergodic and nonergodic senses, and linear convergence rates under additional assumptions. Some preliminary numerical results are reported to demonstrate the efficiency of the ALM with full Jacobian decomposition and LQP regularization.2010 Mathematics Subject Classification. 90C25, 90C33, 65K05.

Iteration-complexity of a Jacobi-type non-Euclidean ADMM for multi-block linearly constrained nonconvex programs

“…Since then several papers have obtained iteration-complexity results for various ADMM variants (see for example [2,9,12,16,18,3,5,11,14,25,33]). Multiblock ADMM variants have also been extensively studied (see for example [15,19,26,27,28,17,4,37,23]). In particular, papers [17,4,37,23] study the convergence and/or complexity of Jacobi-type ADMM variants.…”

“…Multiblock ADMM variants have also been extensively studied (see for example [15,19,26,27,28,17,4,37,23]). In particular, papers [17,4,37,23] study the convergence and/or complexity of Jacobi-type ADMM variants.…”