A Heuristic Fast Gradient Descent Method for Unimodal Optimization

Abstract

The known gradient descent optimization methods applied to convex functions are using the gradient's magnitude in order to adaptively determine the current step size. The paper is presenting a new heuristic fast gradient descent (HFGD) approach, which uses the change in gradient's direction in order to adaptively determine the current step size. The new approach can be applied to solve classes of unimodal functions more general than the convex functions (e.g., quasi-convex functions), or as a local optimization method in multimodal optimization. Testing conducted on a testbed of 16 test functions showed an overall much better efficiency and an overall better success rate of the proposed HFGD method when compared to other three known first order gradient descent methods.

Further Scalable Test Functions for Multidimensional Continuous Optimization

“…This is an example of a difficult unimodal function having an almost flat region with a very narrow escape path to the minimum point [15]. The 2-dimensional case is presented in Fig.…”

“…This is an example of unimodal function that is not quasi-convex [15], the 2-dimensional case being presented in Fig. 2.…”

Iterative configuration of programmable unitary converter based on few layer redundant multi-plane light conversion

“…Unimodality is typically defined for probability distributions [40]. For a multivariable function f : R N → R, unimodality is defined through the level set L(f, α) = {x|f (x) ≤ α, x ∈ R N } and the convexity of L(f, α) [41,42]. In this paper, we extend this definition to functions on U (N ) by considering the pathconnectedness of L(f, α), as U (N ) is not a convex set.…”