Evolutionary Gradient Descent for Non-convex Optimization

Abstract

Non-convex optimization is often involved in artificial intelligence tasks, which may have many saddle points, and is NP-hard to solve. Evolutionary algorithms (EAs) are general-purpose derivative-free optimization algorithms with a good ability to find the global optimum, which can be naturally applied to non-convex optimization. Their performance is, however, limited due to low efficiency. Gradient descent (GD) runs efficiently, but only converges to a first-order stationary point, which may be a saddle point and thus arbitrarily bad. Some recent efforts have been put into combining EAs and GD. However, previous works either utilized only a specific component of EAs, or just combined them heuristically without theoretical guarantee. In this paper, we propose an evolutionary GD (EGD) algorithm by combining typical components, i.e., population and mutation, of EAs with GD. We prove that EGD can converge to a second-order stationary point by escaping the saddle points, and is more efficient than previous algorithms. Empirical results on non-convex synthetic functions as well as reinforcement learning (RL) tasks also show its superiority.

Optimal control under nonconvexity: A generalized Hamiltonian approach

“…27 GD and EA can be combined in evolutionary gradient descent (EGD) methods. 28 EGD methods benefit from the merits of both GA and EA: they work well under nonconvexity (by escaping local solutions and variable-specific saddle points) while converging faster than EA methods. 28 In general, the speed to convergence depends on the nature of the optimization problem and the specifics of the iteration scheme in ( 14) and (15).…”

“…28 EGD methods benefit from the merits of both GA and EA: they work well under nonconvexity (by escaping local solutions and variable-specific saddle points) while converging faster than EA methods. 28 In general, the speed to convergence depends on the nature of the optimization problem and the specifics of the iteration scheme in ( 14) and (15). In this context, knowing the structure of the optimization problem can help speed up convergence.…”

“…EA methods perform reasonably well under nonconvexity: they are more likely to identify a global solution in the presence of local solutions or variable‐specific saddle‐points; but they can be slow to converge 27 . GD and EA can be combined in evolutionary gradient descent (EGD) methods 28 . EGD methods benefit from the merits of both GA and EA: they work well under nonconvexity (by escaping local solutions and variable‐specific saddle points) while converging faster than EA methods 28 …”

“…GD and EA can be combined in evolutionary gradient descent (EGD) methods 28 . EGD methods benefit from the merits of both GA and EA: they work well under nonconvexity (by escaping local solutions and variable‐specific saddle points) while converging faster than EA methods 28 …”

High-Dimensional Dueling Optimization with Preference Embedding

“…Benavoli, Azzimonti, and Piga (2021) improve the optimization performance of PBO by using SkewGP model to fit the preference function but still face the scalability issue. Sui et al (2017) and Xu et al (2020) propose to fix one solution when dueling throughout the optimization process, i.e., kernel-self-sparring (KSS) and comp-GP-UCB. In (Sui et al 2017), KSS substitutes the preference function in PBO with the function whose value is the probability of one solution beating the optimal solution.…”