From differential equation solvers to accelerated first-order methods for convex optimization

Luo, Hao; Chen, Long

doi:10.1007/s10107-021-01713-3

Cited by 24 publications

(12 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following the time rescaling technique from [19,58], for smooth and µ-convex objective f , we propose a primal-dual flow γx = −∇ x L(x, λ), βλ = ∇ λ L(x + x , λ), (1.6) where γ and β are two nonnegative scaling factors that are governed by γ = µ − γ and β = −β, respectively.…”

Section: Resultsmentioning

confidence: 99%

“…As one can add the indicator function of the constraint set to the objective and get rid of the linear constraint in (1.1), the proximal gradient method [7], as well as the accelerated proximal gradient method [6,20,58,59], can be considered. However, they need projections onto the affine constraint set and are not suitable to handle the composite case f = h + g.…”

Section: Related Workmentioning

confidence: 99%

“…For unconstrained problems, there are heavy ball model [3], asymptotically vanishing dynamical (AVD) model [71] and their extensions [2,4,54,80,81]. Besides, Luo and Chen [58] proposed the so-called Nesterov accelerated gradient flow and later generalized it to [19,20,57].…”

Section: Related Workmentioning

confidence: 99%

“…with the initial condition (x(0), λ(0)) = (x 0 , λ 0 ) ∈ Ω, where γ and β are two artificial time rescaling factors and satisfy (cf. [19,58])…”

Section: The Saddle-point Systemmentioning

confidence: 99%

See 3 more Smart Citations

A primal-dual flow for affine constrained convex optimization

Luo

2022

ESAIM: COCV

Self Cite

View full text Add to dashboard Cite

We introduce a novel primal-dual flow for affine constrained convex optimization problems. As a modification of the standard saddle-point system, our primal-dual flow is proved to possess the exponential decay property, in terms of a tailored Lyapunov function. Then two primal-dual methods are obtained from numerical discretizations of the continuous model, and global nonergodic linear convergence rate is established via a discrete Lyapunov function. Instead of solving the subproblem of the primal variable, we apply the semi-smooth Newton iteration to the subproblem with respect to the multiplier, provided that there are some additional properties such as semi-smoothness and sparsity. Especially, numerical tests on the linearly constrained l 1 -l 2 minimization and the total-variation based image denoising model have been provided.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…with the initial condition (x(0), λ(0)) = (x 0 , λ 0 ) ∈ Ω, where γ and β are two artificial time rescaling factors and satisfy (cf. [19,58])…”

Section: The Saddle-point Systemmentioning

confidence: 99%

See 2 more Smart Citations

A primal-dual flow for affine constrained convex optimization

Luo

2022

ESAIM: COCV

Self Cite

View full text Add to dashboard Cite

show abstract

“…Among all BDFs, only BDF2 is A-stable. The A-stability of ODE solvers is related to the acceleration of the first-order optimization method [30].…”

Section: We Call An Ode Solvermentioning

confidence: 99%

Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs

Baker¹,

Xia²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers. These solvers are computationally expensive, requiring the use of tiny step sizes for numerical stability and accuracy guarantees. This paper considers learning neural ODEs using implicit ODE solvers of different orders leveraging proximal operators. The proximal implicit solver consists of inner-outer iterations: the inner iterations approximate each implicit update step using a fast optimization algorithm, and the outer iterations solve the ODE system over time. The proximal implicit ODE solver guarantees superiority over explicit solvers in numerical stability and computational efficiency. We validate the advantages of proximal implicit solvers over existing popular neural ODE solvers on various challenging benchmark tasks, including learning continuous-depth graph neural networks and continuous normalizing flows.

show abstract

Analysis of linear‐quadratic optimal control via a hybrid control lens

Feng,

2024

Asian Journal of Control

View full text Add to dashboard Cite

Linear‐quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is investigated based on the method of hybrid dynamic system and continuous‐time optimization algorithms. We show that the above‐mentioned hybrid system converges exponentially to the optimal feedback gain with any order, and the solution trajectory is Zeno‐free. Then, the forward‐Euler method is utilized to discretize the hybrid system, and a new algorithm called hybrid dynamics‐based gradient descent (HGD) is proposed that could converge to the optimal controller exponentially.

show abstract

From differential equation solvers to accelerated first-order methods for convex optimization

Cited by 24 publications

References 35 publications

A primal-dual flow for affine constrained convex optimization

A primal-dual flow for affine constrained convex optimization

Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs

Analysis of linear‐quadratic optimal control via a hybrid control lens

Contact Info

Product

Resources

About