On the convergence of the direct extension of ADMM for three-block separable convex minimization models with one strongly convex function

Cai, Xingju; Han, Deren; Yuan, Xiaoming

doi:10.1007/s10589-016-9860-y

Cited by 100 publications

(81 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, to discuss the more difficult case where only some of the functions θ i 's are assumed to be strongly convex, we shall also restrict the penalty parameter into certain intervals when discussing the the convergence of e-ADMM (1.3a)-(1.3e) with m ≥ 3. Indeed, as we shall elucidate, the range of β, which is eligible to the case with a generic m, is even larger than those in [1,16] when it reduces to the special case of m = 3. That is, we shall prove the convergence for the e-ADMM (1.3) by requiring only (m − 2) strongly convex functions and a larger range of β for m ≥ 3.…”

Section: Introductionmentioning

confidence: 84%

“…Moreover, we shall establish the worst-case O(1/t) convergence rate in the ergodic sense for m ≥ 3, where t is the iteration counter; and explore some stronger conditions that can ensure the asymptotically linear convergence for m ≥ 3. Thus, compared with existing work in the same category such as [1,4,12,16,17,18], the convergence results in this paper are more general and they are proved under weaker conditions. The rest of this paper is organized as follows.…”

Section: Introductionmentioning

confidence: 89%

“…In addition to the strong convexity of some or all the functions in the objective of (1.1), it is worthwhile to mention that the penalty parameter β in (1.3) should be appropriately restricted to theoretically ensure the convergence, see, e.g., all the work [1,4,12,16,17,18] 1 . According to Theorem 4.1 in [3], even all the functions θ i 's in the objective of (1.1) are strongly convex, the scheme (1.3a)-(1.3e) with m = 3 may be divergent if the penalty parameter β is not well restricted.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization

Tao

Yuan

2017

Adv Comput Math

Self Cite

View full text Add to dashboard Cite

Recently, the alternating direction method of multipliers (ADMM) has found many efficient applications in various areas; and it has been shown that the convergence is not guaranteed when it is directly extended to the multiple-block case of separable convex minimization problems where there are m ≥ 3 functions without coupled variables in the objective. This fact has given great impetus to investigate various conditions on both the model and the algorithm's parameter that can ensure the convergence of the direct extension of ADMM (abbreviated as "e-ADMM"). Despite some results under very strong conditions (e.g., at least (m − 1) functions should be strongly convex) that are applicable to the generic case with a general m, some others concentrate on the special case of m = 3 under the relatively milder condition that only one function is assumed to be strongly convex. We focus on extending the convergence analysis from the case of m = 3 to the more general case of m ≥ 3. That is, we show the convergence of e-ADMM for the case of m ≥ 3 with the assumption of only (m−2) functions being strongly convex; and establish its convergence rates in different scenarios such as the worst-case convergence rates measured by iteration complexity and the asymptotically linear convergence rate under stronger assumptions. Thus the convergence of e-ADMM for the general case of m ≥ 4 is proved; this result seems to be still unknown even though it is intuitive given the known result of the case of m = 3. Even for the special case of m = 3, our convergence results turn out to be more general than the exiting results that are derived specifically for the case of m = 3.

show abstract

Section: Introductionmentioning

confidence: 84%

Section: Introductionmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization

Tao

Yuan

2017

Adv Comput Math

Self Cite

View full text Add to dashboard Cite

show abstract

“…All convex QPs can be written as NLCQP by adding slack variable and/or decomposing a free variable into positive and negative parts. As Q is a huge-sized matrix, it would be beneficial to partition it into block matrices and correspondingly x and A into block variables and block matrices, and then apply BCU methods toward finding a solution to (2).…”

Section: Motivating Examplesmentioning

confidence: 99%

Hybrid Jacobian and Gauss--Seidel Proximal Block Coordinate Update Methods for Linearly Constrained Convex Programming

Xu¹

2018

SIAM J. Optim.

View full text Add to dashboard Cite

Recent years have witnessed the rapid development of block coordinate update (BCU) methods, which are particularly suitable for problems involving large-sized data and/or variables. In optimization, BCU first appears as the coordinate descent method that works well for smooth problems or those with separable nonsmooth terms and/or separable constraints. As nonseparable constraints exist, BCU can be applied under primal-dual settings.In the literature, it has been shown that for weakly convex problems with nonseparable linear constraint, BCU with fully Gauss-Seidel updating rule may fail to converge and that with fully Jacobian rule can converge sublinearly. However, empirically the method with Jacobian update is usually slower than that with Gauss-Seidel rule. To maintain their advantages, we propose a hybrid Jacobian and Gauss-Seidel BCU method for solving linearly constrained multi-block structured convex programming, where the objective may have a nonseparable quadratic term and separable nonsmooth terms. At each primal block variable update, the method approximates the augmented Lagrangian function at an affine combination of the previous two iterates, and the affinely mixing matrix with desired nice properties can be chosen through solving a semidefinite programming. We show that the hybrid method enjoys the theoretical convergence guarantee as Jacobian BCU. In addition, we numerically demonstrate that the method can perform as well as Gauss-Seidel method and better than a recently proposed randomized primal-dual BCU method.

show abstract

“…Cai et al [24] and Li et al [25] have proved the convergence of Extended Alternating Direction Method of Multipliers (e-ADMM) with only one strongly convex function for the case m = 3. Assumption 1.…”

Section: Convergence Analysismentioning

confidence: 99%

Recovery of Corrupted Low-Rank Tensors

Fan¹,

Kuang²

2017

View full text Add to dashboard Cite

This paper studies the problem of recovering low-rank tensors, and the tensors are corrupted by both impulse and Gaussian noise. The problem is well accomplished by integrating the tensor nuclear norm and the l 1 -norm in a unified convex relaxation framework. The nuclear norm is adopted to explore the low-rank components and the l 1 -norm is used to exploit the impulse noise.Then, this optimization problem is solved by some augmented-Lagrangianbased algorithms. Some preliminary numerical experiments verify that the proposed method can well recover the corrupted low-rank tensors.

show abstract

On the convergence of the direct extension of ADMM for three-block separable convex minimization models with one strongly convex function

Cited by 100 publications

References 29 publications

Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization

Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization

Hybrid Jacobian and Gauss--Seidel Proximal Block Coordinate Update Methods for Linearly Constrained Convex Programming

Recovery of Corrupted Low-Rank Tensors

Contact Info

Product

Resources

About