A Derivative-Free Geometric Algorithm for Optimization on a Sphere

Abstract

Optimization on a unit sphere finds crucial applications in science and engineering. However, derivatives of the objective function may be difficult to compute or corrupted by noises, or even not available in many applications. Hence, we propose a Derivative-Free Geometric Algorithm (DFGA) which, to the best of our knowledge, is the first derivative-free algorithm that takes trust region framework and explores the spherical geometry to solve the optimization problem with a spherical constraint. Nice geometry of the spherical surface allows us to pursue the optimization at each iteration in a local tangent space of the sphere. Particularly, by applying Householder and Cayley transformations, DFGA builds a quadratic trust region model on the local tangent space such that the local optimization can essentially be treated as an unconstrained optimization. Under mild assumptions, we show that there exists a subsequence of the iterates generated by DFGA converging to a stationary point of this spherical optimization. Furthermore, under the Łojasiewicz property, we show that all the iterates generated by DFGA will converge with at least a linear or sublinear convergence rate. Our numerical experiments on solving the spherical location problems, subspace clustering and image segmentation problems resulted from hypergraph partitioning, indicate DFGA is very robust and efficient for solving optimization on a sphere without using derivatives.

A Derivative-free Trust-region Method for Optimization on the Ellipsoid

“…For example, in the geography and atmospheric physical sciences, we pay attention to the objective function in many cases on the earth's surface, which exactly leads to an optimization problem on an approximate ellipsoid in ℜ 3 . Besides, the optimization problems on the unit sphere [14] (a special case of the ellipsoid) in ℜ n are the simplest (matrix) optimization problems with the orthogonality constraints. Therefore, the ideas, techniques and methods for the optimization on the unit sphere can be extended to the optimization problems with the orthogonality constraints.…”