Line Search Algorithms for Locally Lipschitz Functions on Riemannian Manifolds

Hosseini, Seyedehsomayeh; Huang, Wen; Yousefpour, Rohollah

doi:10.1137/16m1108145

Cited by 59 publications

(40 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As subsolver for RALM and the smoothing methods REPMS, we use LRBFGS: a Riemannian, limited-memory BFGS [36] as implemented in Manopt [18]. For REPMSD, we use a minimum-norm tangent vector in the extended subgradient, then we use a type of LRBFGS inverse Hessian approximation for Hessian updates to choose the update direction [33]. For these limited-memory subsolvers, we let the memory be 30, the maximum number of iterations be 200, and the minimum step size be 10 −10 .…”

Section: Dataset Name Number Of Data Features Clustersmentioning

confidence: 99%

Simple Algorithms for Optimization on Riemannian Manifolds with Constraints

Liu

Boumal

2019

Appl Math Optim

View full text Add to dashboard Cite

We consider optimization problems on manifolds with equality and inequality constraints. A large body of work treats constrained optimization in Euclidean spaces. In this work, we consider extensions of existing algorithms from the Euclidean case to the Riemannian case. Thus, the variable lives on a known smooth manifold and is further constrained. In doing so, we exploit the growing literature on unconstrained Riemannian optimization. For the special case where the manifold is itself described by equality constraints, one could in principle treat the whole problem as a constrained problem in a Euclidean space. The main hypothesis we test here is whether it is sometimes better to exploit the geometry of the constraints, even if only for a subset of them. Specifically, this paper extends an augmented Lagrangian method and smoothed versions of an exact penalty method to the Riemannian case, together with some fundamental convergence results. Numerical experiments indicate some gains in computational efficiency and accuracy in some regimes for minimum balanced cut, non-negative PCA and k-means, especially in high dimensions.

show abstract

Section: Dataset Name Number Of Data Features Clustersmentioning

confidence: 99%

Simple Algorithms for Optimization on Riemannian Manifolds with Constraints

Liu

Boumal

2019

Appl Math Optim

View full text Add to dashboard Cite

show abstract

“…A concise survey of Newton based methods can be consulted in [14]. Since step lengths in (20) are based on first or second order Taylor polynomials, the step size can be chosen via line search [15] and/or trust region [16] methods. Thus, we can ensure global convergence of optimization methods to stationary points of the cost function (1).…”

Section: Line Search Optimization Methodsmentioning

confidence: 99%

A Random Line-Search Optimization Method via Modified Cholesky Decomposition for Non-linear Data Assimilation

Niño-Ruiz

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper proposes a line-search optimization method for non-linear data assimilation via random descent directions. The iterative method works as follows: at each iteration, quadratic approximations of the Three-Dimensional-Variational (3D-Var) cost function are built about current solutions. These approximations are employed to build sub-spaces onto which analysis increments can be estimated. We sample search-directions from those sub-spaces, and for each direction, a line-search optimization method is employed to estimate its optimal step length. Current solutions are updated based on directions along which the 3D-Var cost function decreases faster. We theoretically prove the global convergence of our proposed iterative method. Experimental tests are performed by using the Lorenz-96 model, and for reference, we employ a Maximum-Likelihood-Ensemble-Filter (MLEF) whose ensemble size doubles that of our implementation. The results reveal that, as the degree of observational operators increases, the use of additional directions can improve the accuracy of results in terms of 2-norm of errors, and even more, our numerical results outperform those of the employed MLEF implementation.

show abstract

“…It is possible to develop new minimizers by making particular choices for the Riemannian operators in Alg 2. We now review the Armijo variant of the Riemannian gradient descent [40], that is a common and probably the simplest choice for an accelerated optimizer on the manifold. Though, many other line-search conditions such as Barzilai-Borwein [47] or strong Wolfe [70] can be used.…”

Section: A2 Riemannian Descent and Line Searchmentioning

confidence: 99%

Probabilistic Permutation Synchronization Using the Riemannian Structure of the Birkhoff Polytope

Birdal

Şimşekli

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We present an entirely new geometric and probabilistic approach to synchronization of correspondences across multiple sets of objects or images. In particular, we present two algorithms: (1) Birkhoff-Riemannian L-BFGS for optimizing the relaxed version of the combinatorially intractable cycle consistency loss in a principled manner, (2) Birkhoff-Riemannian Langevin Monte Carlo for generating samples on the Birkhoff Polytope and estimating the confidence of the found solutions. To this end, we first introduce the very recently developed Riemannian geometry of the Birkhoff Polytope. Next, we introduce a new probabilistic synchronization model in the form of a Markov Random Field (MRF). Finally, based on the first order retraction operators, we formulate our problem as simulating a stochastic differential equation and devise new integrators. We show on both synthetic and real datasets that we achieve high quality multi-graph matching results with faster convergence and reliable confidence/uncertainty estimates.

show abstract

Line Search Algorithms for Locally Lipschitz Functions on Riemannian Manifolds

Cited by 59 publications

References 26 publications

Simple Algorithms for Optimization on Riemannian Manifolds with Constraints

Simple Algorithms for Optimization on Riemannian Manifolds with Constraints

A Random Line-Search Optimization Method via Modified Cholesky Decomposition for Non-linear Data Assimilation

Probabilistic Permutation Synchronization Using the Riemannian Structure of the Birkhoff Polytope

Contact Info

Product

Resources

About