Evaluation Complexity for Nonlinear Constrained Optimization Using Unscaled KKT Conditions and High-Order Models

Abstract

The evaluation complexity of general nonlinear, possibly nonconvex, constrained optimization is analyzed. It is shown that, under suitable smoothness conditions, an-approximate first-order critical point of the problem can be computed in order O(1−2(p+1)/p) evaluations of the problem's functions and their first p derivatives. This is achieved by using a two-phase algorithm inspired by Cartis, Gould, and Toint [SIAM

Second-Order Optimality and Beyond: Characterization and Evaluation Complexity in Convexly Constrained Nonlinear Optimization

“…In the purely unconstrained case, this result recovers known results for q = 1 (first-order criticality for Lipschitz gradients) [46], q = 2 (second-order criticality 7 with Lipschitz Hessians) [18,47] and q = 3 (third-order criticality 8 with Lipschitz continuous third derivative) [1], but extends them to arbitrary order. The results for the convexly constrained case appear to be new and provide in particular the first complexity bound for second-and third-order criticality for such inequality constrained problems.…”

“…In particular, the cost of evaluating any constraint function/derivative possibly defining the convex feasible set F is neglected by the present approach, which must therefore 7 Using (3.34). 8 Using (3.35).…”

Optimality of orders one to three and beyond: Characterization and evaluation complexity in constrained nonconvex optimization

“…Interestingly, an O(ǫ P ǫ −(p+1)/p D min[ǫ D , ǫ P ] −(p+1)/p ) evaluation complexity bound was also proved by Birgin, Gardenghi, Martínez, Santos and Toint in [6] for first-order unscaled, standard KKT conditions and in the least expensive of three cases depending on the degree of degeneracy identifiable by the algorithm 7 . Even if the bounds for the scaled and unscaled cases coincide in order when ǫ P ≤ ǫ D , comparing the two results for first-order critical points is not straightforward.…”

“…In nearly all cases, complexity bounds are given for the task of finding ǫ-approximate firstor (more rarely) second-order critical points, typically using first-or second-order Taylor models of the objective function in a suitable globalization framework such as those that use rust regions or regularization. Notable exceptions are [1] where ǫ-approximate third-order critical points of unconstrained problems are sought, [6,7,[16][17][18] where ǫ-approximate first-order critical points are considered using Taylor models of order higher than two for unconstrained, convexly-constrained, least-squares and equality-constrained problems, respectively, and [19] where general ǫ-approximate q-th order (q ≥ 1) critical points of convexly constrained optimization are analyzed using Taylor models of degree q.…”

“…The bounds derived in [26] for a trust-funnel algorithm also consider a scaled KKT condition and are of the same order. The worst-case evaluation complexity of constrained optimization problems was also recently analyzed in [6], allowing for high-order derivatives and models in a framework inspired by that of both [7] and [15,21]. At variance with these latter references, this analysis considers unscaled approximate first-order critical points in the sense that such points satisfy the standard unscaled KKT conditions with accuracy ǫ P and ǫ D .…”