A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization

Abstract

Let f be a continuous function on R n , and suppose f is continuously differentiable on an open dense subset. Such functions arise in many applications, and very often minimizers are points at which f is not differentiable. Of particular interest is the case where f is not convex, and perhaps not even locally Lipschitz, but whose gradient is easily computed where it is defined. We present a practical, robust algorithm to locally minimize such functions, based on gradient sampling. No subgradient information is required by the algorithm.When f is locally Lipschitz and has bounded level sets, and the sampling radius ǫ is fixed, we show that, with probability one, the algorithm generates a sequence with a cluster point that is Clarke ǫ-stationary. Furthermore, we show that if f has a unique Clarke stationary pointx, then the set of all cluster points generated by the algorithm converges tox as ǫ is reduced to zero.

Cutting-set methods for robust convex optimization with pessimizing oracles

“…When we can evaluate F i and g i in a reasonable time and with reasonable effort, these methods will work, but can be rather slow. Another implementable solution method is the subgradient sampling method introduced in [60,61].…”

Fixed – Order and Structure H∞ Control With Model Based Feedforward for Elastic Web Winding Systems

“…After several years of fundamental research in nonlinear variational analysis, Burke, Lewis and Overton recently designed a nonsmooth, nonconvex, hybrid optimisation algorithm implemented in a publicdomain MATLAB package called HANSO (Hybrid Algorithm for Non-Smooth Optimization). The algorithm mixes in a parametrizable but user-friendly way several optimization techniques, namely quasiNewton updating, bundling and gradient sampling (Burke, et al, 2005). HANSO is at the core of another public-domain Matlab package called HIFOO (H ∞ Fixed Order Optimization) which is tailored at solving fixed-order controller design problems.…”

Positive trigonometric polynomials for strong stability of difference equations

“…The fact that the function is nonsmooth precludes the use of standard optimization procedures. Instead, nonsmooth optimization methods can be used, such as gradient sampling, see [4,22]. However, even though these methods can handle the problem of nonsmoothness, they converge to local extrema as a rule.…”