Optimization Algorithms on Matrix Manifolds

Abstract

3.4 Quotient manifolds 3.4.1 Theory of quotient manifolds 3.4.2 Functions on quotient manifolds 3.4.3 The real projective space RP n−1 3.4.4 The Grassmann manifold Grass(p, n) 3 3.5 Tangent vectors and differential maps CONTENTS ix 5.3 Riemannian connection 5.3.1 Symmetric connections 5.3.2 Definition of the Riemannian connection 5.3.3 Riemannian connection on Riemannian submanifolds 5.3.4 Riemannian connection on quotient manifolds 5.4 Geodesics, exponential mapping, and parallel translation 5.5 Riemannian Hessian operator 5.6 Second covariant derivative* 5.7 Notes and references 6. Newton's Method 6.1 Newton's method on manifolds 6.2 Riemannian Newton method for real-valued functions 6.3 Local convergence 6.3.1 Calculus approach to local convergence analysis 6.4 Rayleigh quotient algorithms 6.4.1 Rayleigh quotient on the sphere 6.4.2 Rayleigh quotient on the Grassmann manifold 6.4.3 Generalized eigenvalue problem 6.4.4 The nonsymmetric eigenvalue problem 6.4.5 Newton with subspace acceleration: Jacobi-Davidson 6.5 Analysis of Rayleigh quotient algorithms 6.5.1 Convergence analysis 6.5.2 Numerical implementation 6.6 Notes and references 7. Trust-Region Methods 7.1 Models 7.1.1 Models in R n 7.1.2 Models in general Euclidean spaces 7.1.3 Models on Riemannian manifolds 7.

Riemannian BFGS Algorithm with Applications

“…A brief introduction to the area can be found in [1] in this volume, and we refer to [3] and the many references therein for more details. Optimization on manifolds finds applications in two broad classes of situations: classical equality-constrained optimization problems where the constraints specify a submanifold of R n ; and problems where the objective function has continuous invariance properties that we want to eliminate for various reasons, e.g., efficiency, consistency, applicability of certain convergence results, and avoiding failure of certain algorithms due to degeneracy.…”

Exponential Barycenters of the Canonical Cartan Connection and Invariant Means on Lie Groups

“…in [62,65,32,21,1] to cite just a few. The iteration we investigate in this paper does not enter into this family as it does not use the covariant derivative of the vector field of which we are trying to find the zeros, Moreover, we cannot recast it as an optimization problem on a Riemannian manifold, as stated above.…”

“…Newton algorithms on Riemannian manifolds were first proposed in the general context of optimization on manifolds [62,65]. Their convergence has been studied in depth in [47,32,1] to cite just a few of the important works. However, when the goal is to compute the Karcher mean, which is a non-linear least-squares problem, it is more efficient to use the Gauss-Newton variant which does not require the computation of the Hessian function of the Riemannian distance: this avoids implementing the connection.…”

Optimal Data Fitting on Lie Groups: a Coset Approach

“…Here, we use the gradient descent with retractions from [6] on the product of the Stiefel manifold. To compute a gradient we use the Riemannian metric on the Stiefel manifold induced by the Euclidean one on R n×p and equip St(n, p) × St(n, p) with the product metric.…”

“…A descent algorithm on a manifold needs suitable local charts R X which map lines in the tangent space onto curves in the manifold. Here, we choose for the Stiefel manifold the polar decomposition retractions from [6]…”