Block-wise Alternating Direction Method of Multipliers for Multiple-block Convex Programming and Beyond

He, Bingsheng; Yuan, Xiaoming

doi:10.5802/smai-jcm.6

Cited by 36 publications

(28 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the convergence domain K for the stepsizes (τ, s) in (8), shown in Fig. 1, is significantly larger than the domain H given in (6) and the convergence domain in [9,15]. For example, the stepsize s can be arbitrarily close to 5/3 when the stepsize τ is close to −1/3.…”

Section: (4)mentioning

confidence: 93%

“…and σ 1 ∈ (p − 1, +∞), σ 2 ∈ (q − 1, +∞) are two proximal parameters 1 for the regularization terms P k i (·) and Q k j (·). He and Yuan [15] also investigated the above GS-ADMM (7) but restricted the stepsize τ = s ∈ (0, 1), which does not exploit the advantages of using flexible stepsizes given in (8) to improve its convergence.…”

Section: (4)mentioning

confidence: 99%

“…Hence, in the following experiments for GS-ADMM, we adapt GS-ADMM-III with default β = 0.06. Also note that σ 2 = 0, which is not allowed by the algorithms discussed in [15].…”

Section: Gs-admm-i/βmentioning

confidence: 99%

See 2 more Smart Citations

Generalized symmetric ADMM for separable convex optimization

Bai

et al. 2017

Comput Optim Appl

View full text Add to dashboard Cite

The Alternating Direction Method of Multipliers (ADMM) has been proved to be effective for solving separable convex optimization subject to linear constraints. In this paper, we propose a Generalized Symmetric ADMM (GS-ADMM), which updates the Lagrange multiplier twice with suitable stepsizes, to solve the multi-block separable convex programming. This GS-ADMM partitions the data into two group variables so that one group consists of p block variables while the other has q block variables, where p ≥ 1 and q ≥ 1 are two integers. The two grouped variables are updated in a Gauss-Seidel scheme, while the variables within each group are updated in a Jacobi scheme, which would make it very attractive for a big data setting. By adding proper proximal terms to the subproblems, we specify the domain of the stepsizes to guarantee that GS-ADMM is globally convergent with a worst-case O(1/t) ergodic convergence rate. It turns out that our convergence domain of the stepsizes is significantly larger than other convergence domains in the literature. Hence, the GS-ADMM is more flexible and attractive on choosing and using larger stepsizes of the dual variable. Besides, two special cases of GS-ADMM, which allows using zero penalty terms, are also discussed and analyzed. Compared with several state-of-the-art methods, preliminary numerical experiments on solving a sparse matrix minimization problem in the statistical learning show that our proposed method is effective and promising.is the Lagrangian function of the problem (1) with the Lagrange multiplier λ ∈ R n . Then, the ALM procedure for solving (1) can be described as follows:However, ALM does not make full use of the separable structure of the objective function of (1) and hence, could not take advantage of the special properties of the component objective

show abstract

Section: (4)mentioning

confidence: 93%

Section: (4)mentioning

confidence: 99%

See 1 more Smart Citation

Generalized symmetric ADMM for separable convex optimization

Bai

et al. 2017

Comput Optim Appl

View full text Add to dashboard Cite

show abstract

“…With different constraints and distributed variables, ADMM can be applicable in many other fields, such as multiple-block convex programming [24], ADMM for tomography with nonlocal regularizers [25], linear classification [26], and optimal power flow problems [27]. …”

Section: Introductionmentioning

confidence: 99%

ADMM-EM Method forL1-Norm Regularized Weighted Least Squares PET Reconstruction

Teng

Sun

Chen

et al. 2016

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

The L 1-norm regularization is usually used in positron emission tomography (PET) reconstruction to suppress noise artifacts while preserving edges. The alternating direction method of multipliers (ADMM) is proven to be effective for solving this problem. It sequentially updates the additional variables, image pixels, and Lagrangian multipliers. Difficulties lie in obtaining a nonnegative update of the image. And classic ADMM requires updating the image by greedy iteration to minimize the cost function, which is computationally expensive. In this paper, we consider a specific application of ADMM to the L 1-norm regularized weighted least squares PET reconstruction problem. Main contribution is derivation of a new approach to iteratively and monotonically update the image while self-constraining in the nonnegativity region and the absence of a predetermined step size. We give a rigorous convergence proof on the quadratic subproblem of the ADMM algorithm considered in the paper. A simplified version is also developed by replacing the minima of the image-related cost function by one iteration that only decreases it. The experimental results show that the proposed algorithm with greedy iterations provides a faster convergence than other commonly used methods. Furthermore, the simplified version gives a comparable reconstructed result with far lower computational costs.

show abstract

“…For excellent reviews of the ADMM, we refer the reader to, for example, [6][7][8][9][10][11][12][13][14] and the references therein, and also the recent published symmetric ADMM with larger step sizes in [15] which is an allsided work for the two-block separable convex minimization problem. Besides, Goldstein et al [16] developed a Nesterov's accelerated ADMM for problem (2) with partitions:…”

Section: Mathematical Problems In Engineeringmentioning

confidence: 99%

Preconditioned ADMM for a Class of Bilinear Programming Problems

Liang¹,

Bai

2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

We design a novel preconditioned alternating direction method for solving a class of bilinear programming problems, where each subproblem is solved by adding a positive-definite regularization term with a proximal parameter. By the aid of the variational inequality, the global convergence of the proposed method is analyzed and a worst-case O(1/ ) convergence rate in an ergodic sense is established. Several preliminary numerical examples, including the Markowitz portfolio optimization problem, are also tested to verify the performance of the proposed method.

show abstract

Block-wise Alternating Direction Method of Multipliers for Multiple-block Convex Programming and Beyond

Cited by 36 publications

References 35 publications

Generalized symmetric ADMM for separable convex optimization

Generalized symmetric ADMM for separable convex optimization

ADMM-EM Method forL1-Norm Regularized Weighted Least Squares PET Reconstruction

Preconditioned ADMM for a Class of Bilinear Programming Problems

Contact Info

Product

Resources

About