Iteration-complexity of a Jacobi-type non-Euclidean ADMM for multi-block linearly constrained nonconvex programs

Melo, Jefferson G.; Monteiro, Renato D. C.

doi:10.48550/arxiv.1705.07229

Cited by 4 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, researchers have extended the ADMM framework to solve nonconvex multi-block problem (1.1), where [16,25,28,33,42,47,46,66,67,68]. The asymptotical convergence and an iteration complexity of O(ǫ −2 ) are established based on two crucial assumptions on the problem data: (a) g p = 0 and (b) the column space of A p contains the column space of the concatenated matrix…”

Section: Nonconvex Constraints Workmentioning

confidence: 99%

“…In addition to being able to handle nonlinear constraints, SDD-ADMM (p > 1) achieves better iteration complexities under a more general setting. Compared to existing multi-block nonconvex ADMM works [16,25,33,42,47,46,66,67,68], we do not impose restrictive assumptions on problem data (see Section 2.2), and we show that SDD-ADMM obtains an ǫ-stationary solution in O(ǫ −4 ) iterations, which can be further improved to O(ǫ −3 ) under an additional technical assumption. Our results improve the O(ǫ −6 ) and O(ǫ −4 ) iteration complexities established in [28], and match the upper bounds in [63], where a more structured two-block problem is considered.…”

mentioning

confidence: 97%

See 1 more Smart Citation

Dual Descent ALM and ADMM

Sun¹,

Sun²

2021

Preprint

View full text Add to dashboard Cite

In this paper we propose and investigate a new class of dual updates within the augmented Lagrangian framework, where the key feature is to reverse the update direction in the traditional dual ascent. When the dual variable is further scaled by a fractional number, we name the resulting scheme scaled dual descent (SDD), and otherwise, unscaled dual descent (UDD). The novel concept of dual descent sheds new light on the classic augmented Lagrangian framework and brings in more possibilities for algorithmic development. For nonlinear equality-constrained multi-block problems, we propose a new algorithm named SDD-ADMM, which combines SDD with the alternating direction method of multipliers (ADMM). We show that SDD-ADMM finds an ǫ-stationary solution in O(ǫ −4 ) iterations, which can be further improved to O(ǫ −3 ) under an additional technical while verifiable assumption. We also propose UDD-ALM, which applies UDD within the augmented Lagrangian method (ALM), for weakly convex minimization over affine constraints. We show that when a certain regularity condition holds at the primal limit point, UDD-ALM asymptotically converges to a stationary point and finds an ǫ-stationary solution in O(ǫ −2 ) iterations. Both SDD-ADMM and UDD-ALM are single-looped, and our iteration complexities are measured by first-order oracles. SDD-ADMM achieves better iteration complexities for a more challenging setting compared to the ADMM literature, while UDD-ALM complements the best-known result of existing ALM-base algorithms. Furthermore, we demonstrate the potential of UDD-ALM to handle nonlinear constraints by assuming a novel descent solution oracle for the proximal augmented Lagrangian relaxation.

show abstract

Section: Nonconvex Constraints Workmentioning

confidence: 99%

mentioning

confidence: 97%

Dual Descent ALM and ADMM

Sun¹,

Sun²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The alternating direction method of multipliers (ADMM) belongs to the latter class of methods and has seen a recent surge in popularity because of its suitability for distributed computation; e.g., see [8]. However, the vast majority of provably convergent ADMM approaches for solving (1) either exploit convexity in the objective function f t or constraints X t or they are tailored for specific instances of (1); e.g., see [9]- [14].…”

Section: A Literature Reviewmentioning

confidence: 99%

“…We supplement the analysis by showing (empirically) that convergence fails if the proximal weights in the algorithm are not chosen appropriately. The algorithm can be interpreted as a nonconvex constrained extension of the Jacobi algorithms proposed in [14], [27], [29], [30], and as a Jacobi extension of the Gauss-Seidel algorithm for nonconvex problems proposed in [18]. In contrast to the majority of existing approaches for nonconvex problems, we provide a practical and automatic parameter tuning scheme, an open-source Julia implementation (that can be downloaded from https://github.com/exanauts/ProxAL.jl), and an empirical demonstration of convergence on a largescale multi-period optimal power flow test case.…”

Section: B Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

A Globally Convergent Distributed Jacobi Scheme for Block-Structured Nonconvex Constrained Optimization Problems

Subramanyam¹,

Kim²,

Schanen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Motivated by the increasing availability of highperformance parallel computing, we design a distributed parallel algorithm for linearly-coupled block-structured nonconvex constrained optimization problems. Our algorithm performs Jacobitype proximal updates of the augmented Lagrangian function, requiring only local solutions of separable block nonlinear programming (NLP) problems. We provide a cheap and explicitly computable Lyapunov function that allows us to establish global and local sublinear convergence of our algorithm, its iteration complexity, as well as simple, practical and theoretically convergent rules for automatically tuning its parameters. This in contrast to existing algorithms for nonconvex constrained optimization based on the alternating direction method of multipliers that rely on at least one of the following: Gauss-Seidel or sequential updates, global solutions of NLP problems, noncomputable Lyapunov functions, and hand-tuning of parameters. Numerical experiments showcase its advantages for large-scale problems, including the multi-period optimization of a 9000-bus AC optimal power flow test case over 168 time periods, solved on the Summit supercomputer using an open-source Julia code.

show abstract

A two-level distributed algorithm for nonconvex constrained optimization

Sun

2022

Comput Optim Appl

View full text Add to dashboard Cite

This paper aims to develop distributed algorithms for nonconvex optimization problems with complicated constraints associated with a network. The network can be a physical one, such as an electric power network, where the constraints are nonlinear power flow equations, or an abstract one that represents constraint couplings between decision variables of different agents. Despite the recent development of distributed algorithms for nonconvex programs, highly complicated constraints still pose a significant challenge in theory and practice. We first identify some difficulties with the existing algorithms based on the alternating direction method of multipliers (ADMM) for dealing with such problems. We then propose a reformulation that enables us to design a two-level algorithm, which embeds a specially structured three-block ADMM at the inner level in an augmented Lagrangian method framework. Furthermore, we prove the global and local convergence as well as iteration complexity of this new scheme for general nonconvex constrained programs, and show that our analysis can be extended to handle more complicated multi-block inner-level problems. Finally, we demonstrate with computation that the new scheme provides convergent and parallelizable algorithms for various nonconvex applications, and is able to complement the performance of the state-of-the-art distributed algorithms in practice by achieving either faster convergence in optimality gap or in feasibility or both.

show abstract

Iteration-complexity of a Jacobi-type non-Euclidean ADMM for multi-block linearly constrained nonconvex programs

Cited by 4 publications

References 33 publications

Dual Descent ALM and ADMM

Dual Descent ALM and ADMM

A Globally Convergent Distributed Jacobi Scheme for Block-Structured Nonconvex Constrained Optimization Problems

A two-level distributed algorithm for nonconvex constrained optimization

Contact Info

Product

Resources

About