A Brief Introduction to Manifold Optimization

Abstract

Manifold optimization is ubiquitous in computational and applied mathematics, statistics, engineering, machine learning, physics, chemistry and etc. One of the main challenges usually is the non-convexity of the manifold constraints. By utilizing the geometry of manifold, a large class of constrained optimization problems can be viewed as unconstrained optimization problems on manifold. From this perspective, intrinsic structures, optimality conditions and numerical algorithms for manifold optimization are investigated. Some recent progress on the theoretical results of manifold optimization are also presented.

A Nonlinear Eigenvalue Problem Associated with the Sum-of-Rayleigh-Quotients Maximization

“…By viewing the sphere M as a single nonlinear equality constraint, the classical methods for nonlinear programming (see e.g., [17] and the Matlab routine fmincon) can be used for solving (1.2). Viewing M as a simple embedded sub-manifold of R n , the recent developments on the manifold optimization (e.g., [1,7,10,11,25]) provide other efficient ways for (1.2). In particular, the Riemannian trust-region (RTR) method we used in Example 3.1 extends the classical well-known trust-region method (see e.g., [17,Chapter 4]) to a general Riemannian manifold.…”

Estimation and Tracking for Millimeter Wave MIMO Systems Under Phase Noise Problem

“…In this section, we apply manifold optimization technique to obtain a sub-optimal solution of the above problem (16). The background on manifold optimization is illustrated in [57]- [59]. Also, application of this algorithm can be found in [60].…”

Effective Algorithms for Solving Trace Minimization Problem in Multivariate Statistics

“…where sym(M) denotes the symmetric part of M, i.e., sym(M): � ((M + M T )/2). We next introduce several different retraction operators on the Stiefel manifold at the current point X for a given step size t and descent direction ξ (one can refer to [27] for more details).…”

“…Numerical experiments and comparisons with other state-ofthe-art methods indicated that their proposed algorithm is very promising. Another popular second-order algorithm on Stiefel manifold is the Riemannian trust-region (RTR) algorithm [24][25][26][27][28], which has been successfully applied to various applications. In particular, Yang et al [29] presented the RTR algorithm for H 2 model reduction of bilinear systems, where the H 2 error norm is treated as a cost function on the Stiefel manifold.…”