A sequential homotopy method for mathematical programming problems

Abstract

We propose a sequential homotopy method for the solution of mathematical programming problems formulated in abstract Hilbert spaces under the Guignard constraint qualification. The method is equivalent to performing projected backward Euler timestepping on a projected gradient/antigradient flow of the augmented Lagrangian. The projected backward Euler equations can be interpreted as the necessary optimality conditions of a primal-dual proximal regularization of the original problem. The regularized problems are always feasible, satisfy a strong constraint qualification guaranteeing uniqueness of Lagrange multipliers, yield unique primal solutions provided that the stepsize is sufficiently small, and can be solved by a continuation in the stepsize. We show that equilibria of the projected gradient/antigradient flow and critical points of the optimization problem are identical, provide sufficient conditions for the existence of global flow solutions, and show that critical points with emanating descent curves cannot be asymptotically stable equilibria of the projected gradient/antigradient flow, practically eradicating convergence to saddle points and maxima. The sequential homotopy method can be used to globalize any locally convergent optimization method that can be used in a homotopy framework. We demonstrate its efficiency for a class of highly nonlinear and badly conditioned control constrained elliptic optimal control problems with a semismooth Newton approach for the regularized subproblems.

A Flow Perspective on Nonlinear Least-Squares Problems

“…Nonetheless, we can embed the solution of ( 10) with a finite t in an outer loop, in which we sequentially update x 0 by an approximate solution of (10). This leads to a sequential homotopy method for Gauß-Newton methods similar to the one proposed in [24] for inexact Sequential Quadratic Programming methods.…”

A Preconditioned Inexact Active-Set Method for Large-Scale Nonlinear Optimal Control Problems

“…A sequential homotopy method has recently been proposed [48] for the approximate solution of (1.1), where the resulting linear saddle-point systems were numerically solved by direct methods based on sparse matrix decompositions. The aim of this article is to address the challenges that arise when the linear systems are solved only approximately by Krylov-subspace methods and to analyze and leverage novel preconditioners that exploit a multiple saddle-point structure, in particular double saddle-point form, which often arises in optimal control problems with partial differential equation (PDE) constraints [37,55,41].…”

“…The convergence analysis in this paper hinges (instead of using descent arguments for some merit function) on staying in a neighborhood of a suitably defined flow, which can be interpreted as the result of an idealized method with infinitesimal stepsize. Flows of this kind were first used by Davidenko [10] and later extended by various researchers as the basis for a plethora of globalization methods [2,11,12,32,14,7,13,46,47,6,48], often with a focus on affine invariance principles and partly to infinite dimensional problems. However, special care needs to be taken for most of these approaches when solving nonconvex optimization problems, as the Newton flow is attracted to saddle-points and maxima.…”

“…However, special care needs to be taken for most of these approaches when solving nonconvex optimization problems, as the Newton flow is attracted to saddle-points and maxima. Here we extend the sequential homotopy method of [48], which solves a sequence of projected backward Euler steps on a projected gradient/antigradient flow in Hilbert space, to allow for inexact linear system solutions and hence the use of Krylov-subspace methods. Salient properties of the sequential homotopy method are that the difficulties of nonconvexity and constraint degeneracy are handled on the nonlinear level by an implicit regularization similar to regularized/stabilized Sequential Quadratic Programming (see, e.g., [59,26,23,21,22] for the finite dimensional case).…”

“…With B X we denote the open unit ball in X. As in [48], we denote with T (C, x) the tangent cone to a nonempty, closed, convex set C at x ∈ C. We denote with K − the polar cone of a nonempty, closed, convex cone K ⊆ X. We generally use bold notation (such as x) for vectors; regular typeface (such as x) is used for continuous variables.…”