A systematic approach to Lyapunov analyses of continuous-time models in convex optimization

Moucer, Céline; Taylor, Adrien; Bach, Francis

doi:10.48550/arxiv.2205.12772

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performance estimation problem (PEP). From the seminal work of Drori & Teboulle (2014), performance estimation problem (PEP) has been widely used to obtain the worst-case complexity of algorithms, including first-order methods (Kim & Fessler, 2017;Taylor et al, 2017;De Klerk et al, 2017;Kim & Fessler, 2018;Taylor et al, 2018b;Barré et al, 2020;De Klerk et al, 2020;Kim & Fessler, 2021;Abbaszadehpeivasti et al, 2022a;Barré et al, 2022b;Kamri et al, 2022;Rotaru et al, 2022;Gupta et al, 2023), operator splitting methods (Ryu et al, 2020), minimax algorithms (Abbaszadehpeivasti et al, 2021Gorbunov et al, 2022;Zamani et al, 2022), proximal point methods (Gu & Yang, 2020;Kim, 2021;Gu & Yang, 2022;, decentralized methods (Colla & Hendrickx, 2021;, coordinate descent methods (Abbaszadehpeivasti et al, 2022b), and even the continuous-time models (Moucer et al, 2022). PEP also finds the optimal method with optimal worst-case complexity (Drori & Teboulle, 2016;Kim & Fessler, 2016;Drori & Taylor, 2020;Taylor & Drori, 2022;Kim, 2021;Park & Ryu, 2022), and is even used to construct the Lyapunov function for the proof of convergence (Taylor et al, 2018a) and complexity lower bound (Dragomir et al, 2022).…”

Section: Inconsistentmentioning

confidence: 99%

Accelerated Infeasibility Detection of Constrained Optimization and Fixed-Point Iterations

Park¹,

Ryu²

2023

Preprint

View full text Add to dashboard Cite

As first-order optimization methods become the method of choice for solving large-scale optimization problems, optimization solvers based on firstorder algorithms are being built. Such generalpurpose solvers must robustly detect infeasible or misspecified problem instances, but the computational complexity of first-order methods for doing so has yet to be formally studied. In this work, we characterize the optimal accelerated rate of infeasibility detection. We show that the standard fixed-point iteration achieves a O(1/k 2 ) and O(1/k) rates, respectively, on the normalized iterates and the fixed-point residual converging to the infimal displacement vector, while the accelerated fixed-point iteration achieves O(1/k 2 ) and Õ(1/k 2 ) rates. We then provide a matching complexity lower bound to establish that Θ(1/k 2 ) is indeed the optimal accelerated rate.

show abstract

Section: Inconsistentmentioning

confidence: 99%

Accelerated Infeasibility Detection of Constrained Optimization and Fixed-Point Iterations

Park¹,

Ryu²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…This is then reformulated in a sequence of steps to arrive at a semidefinite program. The PEP methodology, first presented in [12], has been extended in a sequence of works that guarantee tightness in each step of the reformulation [28,27], has been adapted as a tool for Lyapunov analysis [26,29,21], and extended to monotone inclusion problems [25]. The IQC methodology is based on integral quadratic constraints from the control literature [19] and uses a simple algorithm representation but lacks tightness guarantees.…”

Section: Introductionmentioning

confidence: 99%

Automated tight Lyapunov analysis for first-order methods

Upadhyaya¹,

Banert²,

Taylor³

et al. 2023

Preprint

View full text Add to dashboard Cite

We present a methodology for establishing the existence of quadratic Lyapunov inequalities for a wide range of first-order methods used to solve convex optimization problems. In particular, we consider i) classes of optimization problems of finite-sum form with (possibly strongly) convex and possibly smooth functional components, ii) first-order methods that can be written as a linear system on state-space form in feedback interconnection with the subdifferentials of the functional components of the objective function, and iii) quadratic Lyapunov inequalities that can be used to draw convergence conclusions. We provide a necessary and sufficient condition for the existence of a quadratic Lyapunov inequality that amounts to solving a small-sized semidefinite program. We showcase our methodology on several first-order methods that fit the framework. Most notably, our methodology allows us to significantly extend the region of parameter choices that allow for duality gap convergence in the Chambolle-Pock method when the linear operator is the identity mapping.

show abstract

A systematic approach to Lyapunov analyses of continuous-time models in convex optimization

Cited by 2 publications

References 0 publications

Accelerated Infeasibility Detection of Constrained Optimization and Fixed-Point Iterations

Accelerated Infeasibility Detection of Constrained Optimization and Fixed-Point Iterations

Automated tight Lyapunov analysis for first-order methods

Contact Info

Product

Resources

About