Second-Order Optimality and Beyond: Characterization and Evaluation Complexity in Convexly Constrained Nonlinear Optimization

Abstract

High-order optimality conditions for convexly constrained nonlinear optimization problems are analysed. A corresponding (expensive) measure of criticality for arbitrary order is proposed and extended to define high-order -approximate critical points. This new measure is then used within a conceptual trust-region algorithm to show that if derivatives of the objective function up to order q ≥ 1 can be evaluated and are Lipschitz continuous, then this algorithm applied to the convexly constrained problem needs at most O( −(q+1) ) evaluations of f and its derivatives to compute an -approximate qth-order critical point. This provides the first evaluation complexity result for critical points of arbitrary order in nonlinear optimization. An example is discussed, showing that the obtained evaluation complexity bounds are essentially sharp.

Sharp Worst-Case Evaluation Complexity Bounds for Arbitrary-Order Nonconvex Optimization with Inexpensive Constraints

“…(If the p-th derivative is assumed to be Lispchitz continuous, the bound becomes O(ǫ − p+1 p−q+1 ).) This bound matches the best known lower bounds for first-and second-order, and improves on the bound in O(ǫ −(q+1) ) given by [11]. We then show that this bound is sharp in order for unconstrained problems with Lipschitz continuous p-th derivative by completing and extending the result of [3] in two ways.…”

“…This worst-case evaluation bound generalizes known bounds for q = 1 (see [2]) or q = 2 (see [8]) and significantly improve upon the bounds in O(ǫ −(q+1) ) given by [11] for a more stringent termination rule. It also extends the results obtained in [6] for convexly-constrained problems with q = 1 by allowing the significantly broader class of inexpensive constraints.…”

“…1. When ǫ = 0, (2.12) implies that the complicated path-based necessary optimality conditions derived in [11] do hold. This results from the fact that these latter conditions merely express that the Taylor's model of order q cannot decrease close enough to x along any feasible polynomial path emanating from x, which is clearly the case if x is a global minimizer of the same models in the intersection of the feasible set and a ball of radius δ centered at x.…”

“…. , q} (2.14) was used in [11] instead of (2.12). Our main reason to prefer (2.12) is the following.…”

“…Complexity for obtaining ǫ-approximate second-order-necessary unconstrained minimizers was considered in [19,5], where a bound of O(ǫ −3 ) evaluations was proved to obtain an ǫ-second-order-necessary minimizer using a Taylor's model of degree two, and a bound of O(ǫ − p+1 p−1 ) evaluations was shown in [8] for the case where a Taylor model of degree p is used. Defining q-th-order-necessary minimizers for q > 2 was considered in [11], where the difficulty of stating and verifying necessary optimality was discussed. In particular, it was concluded in this latter reference that defining and computing ǫ-approximate q-thorder-necessary minimizers for q > 2 is likely to remain elusive, essentially because of the nonlinearity and lack of continuity of the kernels of the derivatives involved.…”