Robust and structure exploiting optimization algorithms: An integral quadratic constraint approach

Michalowsky, Simon; Scherer, Carsten W.; Ebenbauer, Christian

doi:10.48550/arxiv.1905.00279

Cited by 6 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We follow the introduction and notation on discretetime IQCs in [11,16,20]. Generally, Let P be a linear, bounded, self-adjoint operator.…”

Section: Data-driven Integral Quadratic Constraintsmentioning

confidence: 99%

See 2 more Smart Citations

Determining optimal input-output properties: A data-driven approach

Koch¹,

Berberich²,

Köhler³

et al. 2020

Preprint

View full text Add to dashboard Cite

Due to their relevance in systems analysis and (robust) controller design, we consider the problem of determining controltheoretic system properties of an a priori unknown system from data only. More specifically, we introduce a necessary and sufficient condition for a discrete-time linear time-invariant system to satisfy a given integral quadratic constraint (IQC) over a finite time horizon using only one input-output trajectory of finite length. Furthermore, for certain classes of IQCs, we provide convex optimization problems in form of semidefinite programs (SDPs) to retrieve the optimal, i.e. the tightest, system property description that is satisfied by the unknown system. Finally, we provide bounds on the difference between finite and infinite horizon IQCs and illustrate the effectiveness of the proposed scheme in a variety of simulation studies including noisy measurements and a high dimensional system.

show abstract

“…We follow the introduction and notation on discretetime IQCs in [11,16,20]. Generally, Let P be a linear, bounded, self-adjoint operator.…”

Section: Data-driven Integral Quadratic Constraintsmentioning

confidence: 99%

“…Special attention must then be given to the choice of ν. For more details on handling acausal multipliers by Toeplitz matrices, the reader is referred to [20].…”

Section: Proof (I)mentioning

confidence: 99%

See 1 more Smart Citation

Determining optimal input-output properties: A data-driven approach

Koch¹,

Berberich²,

Köhler³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Important contributions were made by Bubeck et al (2015), who proposed an accelerated algorithm that has a geometric interpretation, by Allen-Zhu and Orecchia (2014), who show that coupling of gradient and mirror descent can lead to acceleration, and Lessard et al (2016), who propose a general control-theoretic analysis framework. The framework has subsequently been extended and refined, for example by Michalowsky et al (2019), who analyzed and quantified robustness and convergence trade-offs. Other work includes Diakonikolas and Orecchia (2018), who unify the analysis of first-order methods by imposing certain decay conditions, and Scieur et al (2017), who interpret the accelerated gradient method as a multi-step discretization of gradient flow.…”

Section: Introductionmentioning

confidence: 99%

Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives

Muehlebach¹,

Jordan²

2020

Preprint

View full text Add to dashboard Cite

We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their initial conditions, to provide a simple characterization of convergence rates. In many cases, closed-form expressions are obtained that relate algorithm parameters to the convergence rate. The analysis encompasses discrete time and continuous time, as well as time-invariant and time-variant formulations, and is not limited to a convex or Euclidean setting. In addition, the article rigorously establishes why symplectic discretization schemes are important for momentum-based optimization algorithms, and provides a characterization of algorithms that exhibit accelerated convergence.

show abstract

“…The key idea is to exploit structural features of linear and nonlinear terms and utilize theory and techniques from stability analysis of nonlinear dynamical systems to study properties of optimization algorithms. This approach provides new methods for studying not only convergence rate but also robustness of optimization routines [32][33][34][35] and can lead to new classes of algorithms that strike a desired tradeoff between the speed and robustness.…”

Section: Introductionmentioning

confidence: 99%

Proximal gradient flow and Douglas-Rachford splitting dynamics: global exponential stability via integral quadratic constraints

Hassan-Moghaddam¹,

Jovanović²

2019

Preprint

View full text Add to dashboard Cite

Many large-scale and distributed optimization problems can be brought into a composite form in which the objective function is given by the sum of a smooth term and a nonsmooth regularizer. Such problems can be solved via a proximal gradient method and its variants, thereby generalizing gradient descent to a nonsmooth setup. In this paper, we view proximal algorithms as dynamical systems and leverage techniques from control theory to study their global properties. In particular, for problems with strongly convex objective functions, we utilize the theory of integral quadratic constraints to prove global exponential stability of the differential equations that govern the evolution of proximal gradient and Douglas-Rachford splitting flows. In our analysis, we use the fact that these algorithms can be interpreted as variable-metric gradient methods on the forwardbackward and the Douglas-Rachford envelopes and exploit structural properties of the nonlinear terms that arise from the gradient of the smooth part of the objective function and the proximal operator associated with the nonsmooth regularizer. We also demonstrate that these envelopes can be obtained from the augmented Lagrangian associated with the original nonsmooth problem and establish conditions for global exponential convergence even in the absence of strong convexity.

show abstract

Robust and structure exploiting optimization algorithms: An integral quadratic constraint approach

Cited by 6 publications

References 0 publications

Determining optimal input-output properties: A data-driven approach

Determining optimal input-output properties: A data-driven approach

Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives

Proximal gradient flow and Douglas-Rachford splitting dynamics: global exponential stability via integral quadratic constraints

Contact Info

Product

Resources

About