Fixed-Time Stable Gradient Flows: Applications to Continuous-Time Optimization

Abstract

In this paper, continuous-time optimization methods are studied and a novel gradient-flow scheme is designed that yields convergence to the optimal point of the convex objective function in a fixed time from any given initial point. It is shown that the solutions of the modified gradient flow exist and are unique under certain regularity conditions on the objective function, and Lyapunov-based analysis is used to show fixed-time convergence. The unconstrained optimization problem is considered under two different assumptions, namely, strong-convexity and gradient-dominance, and fixed-time convergence is proven for both cases. Next, a modified Newton's method is presented that exhibits fixed-time convergence under some mild conditions on the objective function. Then, a method for solving convex optimization problems with linear equality constraints is proposed that yields convergence to the optimal point in fixed time. The constrained optimization problems are formulated as min-max problems and a novel method of computing the optimal solution is proposed using the Lagrangian dual. Finally, the general min-max problem is considered and a modified scheme for the saddle-point dynamics is proposed so that the optimal solution can be obtained in fixed time. Numerical illustrations are included to corroborate the efficacy of the proposed methods.

Fixed-Time Extremum Seeking

“…In order to analyze system (21), we can follow the ideas of [3] by considering the Lyapunov function…”

“…Remark 2. Unlike the model-based fixed-time gradient dynamics of [3] and [35], the FTGES dynamics are model-free and only need measurements of the cost function φ. Because of this, their stability properties are highly dependent on an appropriate ordered tuning of the parameters (ε 2 a, 1 ).…”

“…In practice, this compact set can be taken arbitrarily large by choosing a large η, and it is used only for the purpose of analysis in order to apply averaging results for nonsmooth singularly perturbed systems. In (10), the optimizing state u is governed by learning dynamics that are designed to approximate a gradient flow with fixed-time convergence properties [3]. However, said approximation requires suitable tuning of the parameters (ε 1 , ε 2 , k, q 1 , q 2 , κ, a) that characterize the closed-loop system.…”

“…These algorithms guarantee convergence to the desired target in a fixed time that is finite and independent of the initial conditions. Algorithms with fixed time stability properties have been recently studied in [29], [30], [31], [32], [33], [3] and [34]. As shown in [31] and [33], this type of convergence property can be established via Lyapunov functions for a class of continuous-time dynamical systems, which has opened the door to novel optimization algorithms characterized by continuous vector fields with non-asymptotic convergence properties; e.g., [3] and [35].…”

“…Algorithms with fixed time stability properties have been recently studied in [29], [30], [31], [32], [33], [3] and [34]. As shown in [31] and [33], this type of convergence property can be established via Lyapunov functions for a class of continuous-time dynamical systems, which has opened the door to novel optimization algorithms characterized by continuous vector fields with non-asymptotic convergence properties; e.g., [3] and [35]. However, to the best of our knowledge, all existing optimization algorithms with fixed-time stability properties are model-based and require access to the first or second derivatives of the cost functions.…”