Simple Algorithms for Optimization on Riemannian Manifolds with Constraints

Abstract

We consider optimization problems on manifolds with equality and inequality constraints. A large body of work treats constrained optimization in Euclidean spaces. In this work, we consider extensions of existing algorithms from the Euclidean case to the Riemannian case. Thus, the variable lives on a known smooth manifold and is further constrained. In doing so, we exploit the growing literature on unconstrained Riemannian optimization. For the special case where the manifold is itself described by equality constraints, one could in principle treat the whole problem as a constrained problem in a Euclidean space. The main hypothesis we test here is whether it is sometimes better to exploit the geometry of the constraints, even if only for a subset of them. Specifically, this paper extends an augmented Lagrangian method and smoothed versions of an exact penalty method to the Riemannian case, together with some fundamental convergence results. Numerical experiments indicate some gains in computational efficiency and accuracy in some regimes for minimum balanced cut, non-negative PCA and k-means, especially in high dimensions.

Manifold Optimization for High-Accuracy Spatial Location Estimation Using Ultrasound Waves

“…More specifically, while Subsection IV-A recalls all necessary manifold terminology, definitions, and notations, Subsection IV-B illustrates the design of a Riemannian optimization algorithm and recalls relevant convergence results. For more details on Riemannian optimization we refer the reader to [32], [33], and [34].…”

“…We numerically evaluate the CCRB, given by (34), under different SNR scenarios and compare the performance of our algorithm against these bounds in the Results section.…”

A Brief Introduction to Manifold Optimization

“…Another class of methods is based on operator-splitting techniques. Some variants of the alternating direction method of multipliers (ADMM) are studied in [56,53,90,107,13,60].…”

Riemannian Newton and trust‐region algorithms for analytic rotation in exploratory factor analysis

“…In principle, the problems specified by equations (7) and (8) can be solved by Lagrangian methods for equality‐constrained nonlinear optimization (e.g., Conn, Gould, & Toint, 1991). In cases where equality constraints yield manifolds with well‐known geometric properties, however, recasting constrained Euclidean optimization as the equivalent unconstrained manifold optimization not only results in simple and efficient algorithms (e.g., Liu & Boumal, 2019) but also offers neat geometric interpretations thereof. For example, Jennrich (2001, 2002) derived the stopping rule for the GP algorithm via a Lagrangian argument, which coincides with a more geometric interpretation that the Riemannian gradient of the objective function should be sufficiently close to zero.…”