Survey of derivative-free optimization

Abstract

In this survey paper we present an overview of derivative-free optimization, including basic concepts, theories, derivative-free methods and some applications. To date, there are mainly three classes of derivative-free methods and we concentrate on two of them, they are direct search methods and model-based methods. In this paper, we first focus on unconstrained optimization problems and review some classical direct search methods and model-based methods in turn for these problems. Then, we survey a number of derivativefree approaches for problems with constraints, including an algorithm we proposed for spherical optimization recently.

A survey of gradient methods for solving nonlinear optimization

“…Properties of Hessian matrix which appear in distributed gradient-based multi-agent control systems was considered in [117]. A survey of derivative-free optimization methods was given in [127]. An application of unconstrained optimization in solving the risk probability was presented in [76].…”

Diffusion-Free Neural Network-Based Algorithm For Hyperbolic Equations

“…The way of finding the optimal point only by function values, without gradient information is called derivative-free optimization. The first way is to use heuristic algorithms that worked well, including clas-sical simulated annealing arithmetic, genetic algorithms, ant colony algorithms, and particle swarm optimization [36]. They all yield global approximate solution but have weak theoretical support.…”