Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Abstract

In this chapter, we identify fundamental geometric structures that underlie the problems of sampling, optimisation, inference and adaptive decision-making. Based on this identification, we derive algorithms that exploit these geometric structures to solve these problems efficiently. We show that a wide range of geometric theories emerge naturally in these fields, ranging from measure-preserving processes, information divergences, Poisson geometry, and geometric integration. Specifically, we explain how (i) leveraging the symplectic geometry of Hamiltonian systems enable us to construct (accelerated) sampling and optimisation methods, (ii) the theory of Hilbertian subspaces and Stein operators provides a general methodology to obtain robust estimators, (iii) preserving the information geometry of decision-making yields adaptive agents that perform active inference. Throughout, we emphasise the rich connections between these fields; e.g., inference draws on sampling and optimisation, and adaptive decision-making assesses decisions by inferring their counterfactual consequences. Our exposition provides a conceptual overview of underlying ideas, rather than a technical discussion, which can be found in the references herein.

Particular flows and attracting sets: A comment on "How particular is the physics of the Free Energy Principle?" by Aguilera, Millidge, Tschantz and Buckley

“…This solenoidal coupling helps particular states accelerate their flow towards (and maintain their proximity to) the free energy minimum. This is in line with investigations into the importance of solenoidal coupling in augmenting the speed of convergence of diffusions to their steady-state [8][9][10][11], and hints at the importance of the 'canonical' or 'normal' form of the flow introduced in previous FEP literature [4,6,12,13].…”

On Bayesian Mechanics: A Physics of and by Beliefs

“…A central application of this formalism is active inference, where the path of active states is a minimiser of expected free energy [66]. 14 We can formulate active states as minimisers of an action,…”

“…Indeed, the gauge-theoretic view that we introduced here is precisely that this is possible because p(x) is covariant on J(x). Future work should extend this to non-stationary regimes; and has, to a limited extent, already begun [66], where the mode and corresponding vertical flow changes direction in time, introducing a continuous interpretation of this iterated inference. Indeed, the suggestion of a continuous view of marginal beliefs along a path can be derived from the principle of maximum path entropy or maximum calibre [51], which we conjecture is an attractive foundation for extensions of the technology of the FEP to genuine non-equilibria.…”

Modelling non-reinforced preferences using selective attention

“…Planning objective for learning preferences. Following Sajid et al (2021b), we substitute the planner with the expected free energy (G) Barp et al, 2022) augmented with conjugate priors to allow for preference learning over time:…”

“…Therefore, in the absence of non-reinforced preferences, or whilst learning them, intrinsic motivation contextualises agent's interactions with the environment in a way that depends upon its posterior beliefs about latent environmental states (Barto, 2013;Ryan & Deci, 2000). Here, actions are selected by sampling from the distribution P (π) = arg max(−G(π)) (Barp et al, 2022).…”