Optimization on manifolds: A symplectic approach

França, Guilherme; Barp, Alessandro; Jordan, Michael I.

doi:10.48550/arxiv.2107.11231

Cited by 5 publications

(18 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we will see, such geometric integrators can be constructed by leveraging the shadow Hamiltonian property of symplectic methods on higher-dimensional conservative Hamiltonian systems [9] (see also [98,99]). This holds not only on R 2d but on general settings, namely on arbitrary smooth manifolds [9,10].…”

Section: Rate-matching Integrators For Smooth Optimisationmentioning

confidence: 81%

“…In the context of optimisation this means respecting stability and rates of convergence. This was first demonstrated in [9] and further extended in [10]; our following discussion will be based on these works.…”

Section: Principle Of Geometric Integrationmentioning

confidence: 82%

“…Differential geometry plays a fundamental role in applied mathematics, statistics, and computer science, including numerical integration [1][2][3][4][5], optimisation [6][7][8][9][10][11], sampling [12][13][14][15][16], statistics on spaces with deep learning [17,18], medical imaging and shape methods [19,20], interpolation [21], and the study of random maps [22], to name a few. Of particular relevance to this chapter is information geometry, i.e., the differential geometric treatment of smooth statistical manifolds, whose origin stems from a seminal article by Rao [23] who introduced the Fisher metric tensor on parametrised statistical models, and thus a natural Riemannian geometry that was later observed to correspond to an infinitesimal distance with respect to the Kullback-Leibler (KL) divergence [24].…”

Section: Introductionmentioning

confidence: 99%

“…More recently, it has been proved that a generalisation of symplectic integrators to dissipative Hamiltonian systems is indeed able to preserve rates of convergence and stability [9], which are the main properties of interest for optimisation. Followup work [10] extended this approach, enabling optimisation on manifolds and problems with constraints. There is also a tight connection between optimisation on the space of measures and sampling which dates back to [50,51]; we will revisit these ideas in relation to dissipative Hamiltonian systems.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Barp,

Da Costa,

França

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this chapter, we identify fundamental geometric structures that underlie the problems of sampling, optimisation, inference and adaptive decision-making. Based on this identification, we derive algorithms that exploit these geometric structures to solve these problems efficiently. We show that a wide range of geometric theories emerge naturally in these fields, ranging from measure-preserving processes, information divergences, Poisson geometry, and geometric integration. Specifically, we explain how (i) leveraging the symplectic geometry of Hamiltonian systems enable us to construct (accelerated) sampling and optimisation methods, (ii) the theory of Hilbertian subspaces and Stein operators provides a general methodology to obtain robust estimators, (iii) preserving the information geometry of decision-making yields adaptive agents that perform active inference. Throughout, we emphasise the rich connections between these fields; e.g., inference draws on sampling and optimisation, and adaptive decision-making assesses decisions by inferring their counterfactual consequences. Our exposition provides a conceptual overview of underlying ideas, rather than a technical discussion, which can be found in the references herein.

show abstract

Section: Rate-matching Integrators For Smooth Optimisationmentioning

confidence: 81%

Section: Principle Of Geometric Integrationmentioning

confidence: 82%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Barp,

Da Costa,

França

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For both g-convex and g-strongly convex cases, [1] proposed ODEs that can model accelerated methods on Riemannian manifolds given K min and D. [26] extended this result and developed a variational framework. Time-discretization methods for such ODEs on Riemannian manifolds have recently been of considerable interest as well [27][28][29].…”

Section: Related Workmentioning

confidence: 99%

Nesterov Acceleration for Riemannian Optimization

Kim¹,

Yang²

2022

Preprint

View full text Add to dashboard Cite

In this paper, we generalize the Nesterov accelerated gradient (NAG) method to solve Riemannian optimization problems in a computationally tractable manner. The iteration complexity of our algorithm matches that of the NAG method on the Euclidean space when the objective functions are geodesically convex or geodesically strongly convex. To the best of our knowledge, the proposed algorithm is the first fully accelerated method for geodesically convex optimization problems without requiring strong convexity. Our convergence rate analysis exploits novel metric distortion lemmas as well as carefully designed potential functions. We also identify a connection with the continuous-time dynamics for modeling Riemannian acceleration in Alimisis et al. [1] to understand the accelerated convergence of our scheme through the lens of continuous-time flows.

show abstract

Bregman dynamics, contact transformations and convex optimization

et al. 2023

View full text Add to dashboard Cite

Recent research on accelerated gradient methods of use in optimization has demonstrated that these methods can be derived as discretizations of dynamical systems. This, in turn, has provided a basis for more systematic investigations, especially into the geometric structure of those dynamical systems and their structure-preserving discretizations. In this work, we introduce dynamical systems defined through a contact geometry which are not only naturally suited to the optimization goal but also subsume all previous methods based on geometric dynamical systems. As a consequence, all the deterministic flows used in optimization share an extremely interesting geometric property: they are invariant under contact transformations. In our main result, we exploit this observation to show that the celebrated Bregman Hamiltonian system can always be transformed into an equivalent but separable Hamiltonian by means of a contact transformation. This in turn enables the development of fast and robust discretizations through geometric contact splitting integrators. As an illustration, we propose the Relativistic Bregman algorithm, and show in some paradigmatic examples that it compares favorably with respect to standard optimization algorithms such as classical momentum and Nesterov’s accelerated gradient.

show abstract

Optimization on manifolds: A symplectic approach

Cited by 5 publications

References 30 publications

Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Nesterov Acceleration for Riemannian Optimization

Bregman dynamics, contact transformations and convex optimization

Contact Info

Product

Resources

About