Pulling back information geometry

Abstract

Latent space geometry has shown itself to provide a rich and rigorous framework for interacting with the latent variables of deep generative models. The existing theory, however, relies on the decoder being a Gaussian distribution as its simple reparametrization allows us to interpret the generating process as a random projection of a deterministic manifold. Consequently, this approach breaks down when applied to decoders that are not as easily reparametrized. We here propose to use the Fisher-Rao metric associated with the space of decoder distributions as a reference metric, which we pull back to the latent space. We show that we can achieve meaningful latent geometries for a wide range of decoder distributions for which the previous theory was not applicable, opening the door to 'black box' latent geometries. * Equal contribution.Preprint. Under review.

Several Functions Originating from Fisher–Rao Geometry of Dirichlet Distributions and Involving Polygamma Functions

“…In the electronic arXiv preprints [1,2] and their formally published version [3], the authors investigated the geometry induced by the Fisher-Rao metric on the parameter space of the Dirichlet distributions in statistics theory, showed that the parameter space is a Hadamard manifold (that is, the manifold is geodesically complete and has negative sectional curvature everywhere), and demonstrated that the Fréchet mean of a set of the Dirichlet distributions is uniquely defined in the geometry. The papers [1][2][3] have been cited in [4][5][6][7][8][9][10][11][12][13][14][15][16]. This means that the papers [1][2][3] have attracted great interest from more and more mathematicians in a short time.…”