Analysis of direct searches for discontinuous functions

Abstract

It is known that the Clarke generalized directional derivative is nonnegative along the limit directions generated by directional directsearch methods at a limit point of certain subsequences of unsuccessful iterates, if the function being minimized is Lipschitz continuous near the limit point.In this paper we generalize this result for discontinuous functions using Rockafellar generalized directional derivatives (upper subderivatives). We show that Rockafellar derivatives are also nonnegative along the limit directions of those subsequences of unsuccessful iterates when the function values converge to the function value at the limit point. This result is obtained assuming that the function is directionally Lipschitz with respect to the limit direction.It is also possible under appropriate conditions to establish more insightful results by showing that the sequence of points generated by these methods eventually approaches the limit point along the locally best branch or step function (when the number of steps is equal to two).The results of this paper are presented for constrained optimization and illustrated numerically.

Worst case complexity of direct search

“…In fact, we prove in Section 3 of this paper that such generating set search (GSS) type methods take at most O( −2 ) iterations to drive the norm of the gradient of the objective function below . Interestingly, this bound is achieved when using a quadratic function of the step size in the sufficient decrease condition, i.e., a function like ρ(t) = ct 2 , c > 0 -which corroborates previous numerical experience [13] where different functions of the form ρ(t) = ct p (with 2 = p > 1) where tested but leading to a worse performance.…”

“…To write the algorithm in general terms we will useρ(·) to either represent a forcing function ρ(·) or the constant, zero function. A relatively minor difference between the presentation below and the one in [6,Chapter 7] (see [13]) is the use ofρ(…”

“…3.1) suff. decrease, non-smooth [13] dense in unit sphere 2)ρ(·) = 0 GSS suff. decrease, smooth [10] any any forcing suff.…”

“…decrease, smooth [10] any any forcing suff. decrease, non-smooth [13] any any forcing of new iterates, which can be simply achieved by selectingρ(·) as a forcing function in Algorithm 2.1. Table 1 summarizes the properties of the set D of directions used by the different variants of this type of direct search.…”

“…Without new 'proof technology', it seems only possible to establish a worst case complexity bound for those direct-search methods based on the acceptance of new iterates by a sufficient decrease condition (using a forcing function of the step size parameter) and when applied to smooth functions. It is interesting to note that direct-search methods which poll using a set of directions dense in the unit sphere allow the derivation of global convergence results for non-smooth functions without specifically requiring the use of a positive spanning set for polling (see [2,13]). However, they seem not capable of offering a complexity bound (even for the smooth case) without polling at a positive spanning set.…”