Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary

Abstract

In this paper we consider the minimization of a continuous function that is potentially not differentiable or not twice differentiable on the boundary of the feasible region. By exploiting an interior point technique, we present first-and second-order optimality conditions for this problem that reduces to classical ones when the derivative on the boundary is available. For this type of problems, existing necessary conditions often rely on the notion of subdifferential or become non-trivially weaker than the KKT condition in the (twice-)differentiable counterpart problems. In contrast, this paper presents a new set of first-and second-order necessary conditions that are derived without the use of subdifferential and reduces to exactly the KKT condition when (twice-)differentiability holds. As a result, these conditions are stronger than some existing ones considered for the discussed minimization problem when only non-negativity constraints are present. To solve for these optimality conditions in the special but important case of linearly constrained problems, we present two novel interior trust-region point algorithms and show that their worstcase computational efficiency in achieving the potentially stronger optimality conditions match the best known complexity bounds. Since this work considers a more general problem than the literature, our results also indicate that best known complexity bounds hold for a wider class of nonlinear programming problems.

A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees

“…However, the firstorder conditions are strictly weaker than those used in the current work as they consist only of feasibility of x along with a scaled gradient condition that is an "unbounded" version of (6c) in whichX is replaced by X. Without additional assumptions on f , the absence of condition (6b) in the optimality conditions implies that sequences of (strictly feasible) points that satisfy the scaled gradient condition may not approach KKT points as ǫ g approaches 0; see [30,Section 2] for a discussion of this issue. Our approximate optimality conditions (6) here do not suffer from these issues, as we show in Section 3.…”

“…Our approximate optimality conditions (6) here do not suffer from these issues, as we show in Section 3. In a follow up to [3], an interior-point method for linear equality and bound constraints was described in [30]. This method, which also achieves an iteration complexity of O(ǫ −3/2 g ) (when ǫ H = ǫ 1/2 g ), applies a constrained secondorder trust-region algorithm to the log-barrier function, with a (potentially) small trust-region radius.…”

“…This method, which also achieves an iteration complexity of O(ǫ −3/2 g ) (when ǫ H = ǫ 1/2 g ), applies a constrained secondorder trust-region algorithm to the log-barrier function, with a (potentially) small trust-region radius. The authors of [30] were more interested in iteration complexity than computational complexity, but we note that each of their subproblems requires evaluation of the Hessian (which in the worst case requires evaluation of n Hessianvector products, where the latter is one of our units of computational complexity), together withÕ(n 3 ) floating point operations associated with performing a bisection scheme to solve the subproblem. These considerations suggest an overall worst-case computational complexity of at least O(nǫ −3/2 g ) for the algorithm of [30].…”

“…The authors of [30] were more interested in iteration complexity than computational complexity, but we note that each of their subproblems requires evaluation of the Hessian (which in the worst case requires evaluation of n Hessianvector products, where the latter is one of our units of computational complexity), together withÕ(n 3 ) floating point operations associated with performing a bisection scheme to solve the subproblem. These considerations suggest an overall worst-case computational complexity of at least O(nǫ −3/2 g ) for the algorithm of [30]. In this paper, we adapt the Newton-CG method of [36] for unconstrained optimization to the problem of minimizing the primal log-barrier function (2), for a small, fixed value of µ.…”

“…The appearance of n in our complexity expressions is an apparently unavoidable consequence of using log-barrier methodology, along with making the mildest possible assumptions on the problem (1) and the algorithm. For example, we do not assume a bounded feasible set or a particular choice of starting point (as in [30]), and we do not assume any specific rate of growth of f as x moves away from the solution set. Still, our computational complexity rates match (for small n) or improve on (for large n) those in [30].…”