Accelerated primal-dual methods for linearly constrained convex optimization problems

Luo, Hao

doi:10.48550/arxiv.2109.12604

Cited by 5 publications

(14 citation statements)

References 64 publications

(132 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proof. The proof of ( 74) is more or less in line with that of the decay estimate in [80,Theorem 4.1]. The key is to start with the identity…”

Section: B Some Sequences and Decay Ratesmentioning

confidence: 59%

“…In [81], Luo proposed a first-order dynamical system and presented a class of primal-dual algorithms for (6) with nonergodic rates. Later in [80], we combined this model with Nesterov accelerated gradient flow [82] and obtained the so-called accelerated primal-dual flow. We also proposed new accelerated primal-dual algorithms and proved nonergodic convergence rates via a discrete Lyapunov function.…”

Section: Dynamical System Approachmentioning

confidence: 99%

“…Applying λk+1 = λ k+1 to (52a) and ( 66), we can obtain the corresponding algorithms that have similar subproblems with (68). Following the spirit of [71,80,81,90], one can call the semi-smooth Newton iteration [36] to solve (68) efficiently provided that it has special structure such as sparsity and semismooth property.…”

Section: The Implicit Discretization Of λmentioning

confidence: 99%

“…and prove the exponential decay E(t) e −t E(0) under the smooth case that both f and g have Lipschitz continuous gradients. Note that our previous accelerated primal-dual flow system proposed in [80,Section 2] can also be applied to the separable case (1) but it treats (x, y) as an entire variable and only involves a single time scaling parameter for F . However, the current model (3a) adopts different scaling factors γ and β respectively for f and g. This not only allows us to handle the partially strongly convex case µ f > 0, µ g = 0 (or µ f = 0, µ g > 0) but also paves the way for designing new primal-dual splitting algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate

Luo¹,

Zhang²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper provides a self-contained ordinary differential equation solver approach for linearly constrained separable convex optimization problems and presents several classes of new accelerated primal-dual splitting methods. A novel dynamical system with built-in time rescaling factors is first introduced and the exponential decay of a tailored Lyapunov function is established. Numerical discretizations of the continuous time model are considered to construct new algorithms for the original optimization problem and also analyzed via a unified discrete Lyapunov function. Specifically, those schemes lead to new proximal alternating direction methods of multipliers with linearizations and extrapolations, and accelerated nonergodic rates are proved respectively for convex and partially strong convex objectives.

show abstract

“…Proof. The proof of ( 74) is more or less in line with that of the decay estimate in [80,Theorem 4.1]. The key is to start with the identity…”

Section: B Some Sequences and Decay Ratesmentioning

confidence: 59%

Section: Dynamical System Approachmentioning

confidence: 99%

Section: The Implicit Discretization Of λmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate

Luo¹,

Zhang²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Attouch et al [4] proposed a temporally rescaled inertial augmented Lagrangian system (TRIALS) with three time-varying parameters (i.e., viscous damping, extrapolation and temporal scaling) to address separable smooth/nonsmooth convex optimization problems with an affine constraint, and presented the fast convergence properties of TRIALS. In addition, He et al [31] and Luo et al [39] further proposed a "second-order"+"first-order" primal-dual dynamical approaches for solving problem (1.7) with accelerated convergence guarantees.…”

mentioning

confidence: 99%

Accelerated Primal-Dual Mirror Dynamical Approaches for Constrained Convex Optimization

Zhao¹,

Liao²,

He³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper investigates two accelerated primal-dual mirror dynamical approaches for smooth and nonsmooth convex optimization problems with affine and closed, convex set constraints. In the smooth case, an accelerated primal-dual mirror dynamical approach (APDMD) based on accelerated mirror descent and primal-dual framework is proposed and accelerated convergence properties of primal-dual gap, feasibility measure and the objective function value along with trajectories of APDMD are derived by the Lyapunov analysis method. Then, we extend APDMD into two distributed dynamical approaches to deal with two types of distributed smooth optimization problems, i.e., distributed constrained consensus problem (DCCP) and distributed extended monotropic optimization (DEMO) with accelerated convergence guarantees. Moreover, in the nonsmooth case, we propose a smoothing accelerated primal-dual mirror dynamical approach (SAPDMD) with the help of smoothing approximation technique and the above APDMD. We further also prove that primal-dual gap, objective function value and feasibility measure of SAPDMD have the same accelerated convergence properties as APDMD by choosing the appropriate smooth approximation parameters. Later, we propose two smoothing accelerated distributed dynamical approaches to deal with nonsmooth DEMO and DCCP to obtain accelerated and efficient solutions. Finally, numerical experiments are given to demonstrate the effectiveness of the proposed accelerated mirror dynamical approaches.

show abstract

Faster augmented Lagrangian method for solving convex optimization problems with linear constraints

Zhang

2024

Preprint

View full text Add to dashboard Cite

A faster augmented Lagrangian method (Faster ALM) with a relaxed parameter ranging from 0 to 2 is introduced in this paper for solving convex optimization problems with equality constraints. The proposed Faster ALM demonstrates a convergence rate of $O\left(1/\sum_{i=0}^{k}a_i\right)$, where ($a_i>0$ is an arbitrary constant), in a non-ergodic sense of the Lagrangian primal-dual gap, the objective function value, and the feasibility measure.To further reduce the computational cost in each iteration, we present the Linearized Faster ALM, which involves linearizing the augmented Lagrangian term while maintaining a relaxed parameter within the range of 0 to 2. The proposed Linearized Faster ALM exhibits a convergence rate of $O(1/k)$ in a non-ergodic sense of the Lagrangian primal-dual gap, the objective function value, and the feasibility measure. MSC Classification: 47H09 , 47H10 , 90C25 , 90C30

show abstract

Accelerated primal-dual methods for linearly constrained convex optimization problems

Cited by 5 publications

References 64 publications

A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate

A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate

Accelerated Primal-Dual Mirror Dynamical Approaches for Constrained Convex Optimization

Faster augmented Lagrangian method for solving convex optimization problems with linear constraints

Contact Info

Product

Resources

About