Optimizing method using frequency annealing: Escape from a local minimum by approximate descent estimation of a low‐pass filter using neighbor data

Abstract

SUMMARYThis paper proposes frequency annealing (FA) as an optimization method which searches for the global optimal solution without using the derivative of the objective function. FA detects the global optimal solution efficiently by smoothing the objective function with a low-pass filter. The gradient of the low-pass filter, which is defined by a convolution integral, is approximated by the value of the objective function at a finite number of evaluation points defined according to the Gaussian probability density function. Thus, a gradient method without using the derivative of the objective function is realized. The effectiveness of the approximation of the low-pass filter gradient is verified by the variance analysis of the approximation value. It is shown that by defining the neighbor points with an adequate probability density function, the approximation accuracy is made independent of the dimension M. The usefulness of the approach is demonstrated by a two-dimensional problem and a simulation of perceptron training. FA can detect the global optimal solution even when the objective function has a local minimum or a discontinuity.