On the optimal order of worst case complexity of direct search

Abstract

The worst case complexity of direct-search methods has been recently analyzed when they use positive spanning sets and impose a sufficient decrease condition to accept new iterates. For smooth unconstrained optimization, it is now known that such methods require at most O(n 2 ϵ −2 ) function evaluations to compute a gradient of norm below ϵ ∈ (0, 1), where n is the dimension of the problem. Such a maximal effort is reduced to O(n 2 ϵ −1 ) if the function is convex. The factor n 2 has been derived using the positive spanning set formed by the coordinate vectors and their negatives at all iterations.In this paper, we prove that such a factor of n 2 is optimal in these worst case complexity bounds, in the sense that no other positive spanning set will yield a better order of n. The proof is based on an observation that reveals the connection between cosine measure in positive spanning and sphere covering.

Positive bases with maximal cosine measure

“…To show that the maximal size is 2n is not trivial, but a short proof using ideas from optimization theory was given by Audet [2]. In [7], the cosine measure of a positive spanning set for R n of size 2n is conjectured to always be less than or equal to 1/ √ n. The present paper shows that this is indeed the case for the maximal positive bases (which is a subset of the positive spanning sets of size 2n).…”

“…The convergence properties of several derivative free algorithms depends on the cosine measure of a positive spanning set, and it is preferable to design the positive B Geir Naevdal gena@norceresearch.no 1 NORCE Norwegian Research Centre, Postboks 22, 5838 Bergen, Norway spanning set having its cosine measure as large as possible. In a recent paper, the problem of determining positive spanning sets with maximal cosine measure is described as "widely open" [7]. The present paper shows that the maximal cosine measure for a positive spanning set for R n of size n + 1 in 1/n.…”

“…Obviously, the results provided here should be complemented. One direction would be to find the maximal cosine measure for positive spanning sets consisting of 2n vectors in R n as discussed in [7]. One can also look for extension of the results provided here to positive bases for R n of size r for n + 1 < r < 2n.…”

Optimality of orders one to three and beyond: Characterization and evaluation complexity in constrained nonconvex optimization

“…If one additionally requests that c(x ǫ + d) ≤ ǫ P , then, from the first part of (10), (28) holds. Also note that, if d exists such that c(x ǫ + d) = 0, x ǫ + d ∈ F and (28) holds, then (29) ensures that (28) holds, then (29) ensures that (30) also holds. Now turn to the case where f (x ǫ ) > t ǫ and j = 2.…”

Chapter 37: Methodologies and Software for Derivative-Free Optimization

“…The factor of n 2 has been proved to be approximately optimal, in a certain sense, in the WCC bounds for the number of function evaluations attained by direct search (see [68]). …”