Convergence Study on the Symmetric Version of ADMM with Larger Step Sizes

In this note, we show a sublinear nonergodic convergence rate for the algorithm developed in [Bai, et al. Generalized symmetric ADMM for separable convex optimization. Comput. Optim. Appl. 70, 129-170 (2018)], as well as its linear convergence under assumptions that the sub-differential of each component objective function is piecewise linear and all the constraint sets are polyhedra. These remaining convergence results are established for the stepsize parameters of dual variables belonging to a special isosceles triangle region, which aims to strengthen our understanding for convergence of the generalized symmetric ADMM.

show abstract

“…Lemma 2.4 Let Q, M, H be given by (8), (11) and (13), respectively. Then, the sequences {w k } and { w k } generated by GS-ADMM satisfy…”

Section: Sublinear Nonergodic Convergence Ratementioning

confidence: 99%

Convergence revisit on generalized symmetric ADMM

Bai

Chang

et al. 2019

Optimization

View full text Add to dashboard Cite

show abstract

“…For excellent reviews of the ADMM, we refer the reader to, for example, [6][7][8][9][10][11][12][13][14] and the references therein, and also the recent published symmetric ADMM with larger step sizes in [15] which is an allsided work for the two-block separable convex minimization problem. Besides, Goldstein et al [16] developed a Nesterov's accelerated ADMM for problem (2) with partitions:…”

Section: Mathematical Problems In Engineeringmentioning

confidence: 99%

“…Moreover, the global convergence and the worst-case O(1/t) convergence rate of the proposed method are analyzed in detail. We begin with a basic lemma given in [15]. …”

Section: Convergence Analysismentioning

confidence: 99%

Preconditioned ADMM for a Class of Bilinear Programming Problems

Liang¹,

Bai

2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

We design a novel preconditioned alternating direction method for solving a class of bilinear programming problems, where each subproblem is solved by adding a positive-definite regularization term with a proximal parameter. By the aid of the variational inequality, the global convergence of the proposed method is analyzed and a worst-case O(1/ ) convergence rate in an ergodic sense is established. Several preliminary numerical examples, including the Markowitz portfolio optimization problem, are also tested to verify the performance of the proposed method.

show abstract

“…More recently, a Symmetric ADMM (S-ADMM) was proposed by He et al [16] for solving the two-block separable convex minimization, where the algorithm performs the following updating scheme:…”

Section: (4)mentioning

confidence: 99%

Generalized symmetric ADMM for separable convex optimization

Bai

et al. 2017

Comput Optim Appl

View full text Add to dashboard Cite

The Alternating Direction Method of Multipliers (ADMM) has been proved to be effective for solving separable convex optimization subject to linear constraints. In this paper, we propose a Generalized Symmetric ADMM (GS-ADMM), which updates the Lagrange multiplier twice with suitable stepsizes, to solve the multi-block separable convex programming. This GS-ADMM partitions the data into two group variables so that one group consists of p block variables while the other has q block variables, where p ≥ 1 and q ≥ 1 are two integers. The two grouped variables are updated in a Gauss-Seidel scheme, while the variables within each group are updated in a Jacobi scheme, which would make it very attractive for a big data setting. By adding proper proximal terms to the subproblems, we specify the domain of the stepsizes to guarantee that GS-ADMM is globally convergent with a worst-case O(1/t) ergodic convergence rate. It turns out that our convergence domain of the stepsizes is significantly larger than other convergence domains in the literature. Hence, the GS-ADMM is more flexible and attractive on choosing and using larger stepsizes of the dual variable. Besides, two special cases of GS-ADMM, which allows using zero penalty terms, are also discussed and analyzed. Compared with several state-of-the-art methods, preliminary numerical experiments on solving a sparse matrix minimization problem in the statistical learning show that our proposed method is effective and promising.is the Lagrangian function of the problem (1) with the Lagrange multiplier λ ∈ R n . Then, the ALM procedure for solving (1) can be described as follows:However, ALM does not make full use of the separable structure of the objective function of (1) and hence, could not take advantage of the special properties of the component objective

show abstract

Convergence Study on the Symmetric Version of ADMM with Larger Step Sizes

Cited by 90 publications

References 25 publications

Convergence revisit on generalized symmetric ADMM

Convergence revisit on generalized symmetric ADMM

Preconditioned ADMM for a Class of Bilinear Programming Problems

Generalized symmetric ADMM for separable convex optimization

Contact Info

Product

Resources

About