Computational Information Geometry in Statistics: Mixture Modelling

Abstract

Abstract. This paper applies the tools of computation information geometry [3] -in particular, high dimensional extended multinomial families as proxies for the 'space of all distributions' -in the inferentially demanding area of statistical mixture modelling. A range of resultant benefits are noted.

Computational Information Geometry in Statistics: Theory and Practice

“…The −1-convex hull of an exponential family is of great interest, mixture models being widely used in many areas of statistical science. In particular, they are explored further in [5]. Here, we simply state the main result, a simple consequence of the total positivity of exponential families [17], that, generically, convex hulls are of maximal dimension.…”

“…Essentially, this works by exploiting the structural properties of information geometry, which are such that all formulae can be expressed in terms of four fundamental building blocks: defined and detailed in Amari [3], these are the +1 and −1 geometries, the way that these are connected via the Fisher information and the foundational duality theorem. Additionally, computational information geometry enables a range of methodologies and insights impossible without it; notably, those deriving from the operational, universal model space, which it affords; see, for example, [4][5][6].…”

Computational Information Geometry in Statistics: Foundations

“…In particular they are explored further in [3] in this volume. Here we simply state the main result, a simple consequence of the total positivity of exponential families [12], that, generically, convex hulls are of maximal dimension.…”

“…In the −1-representation, the log-likelihood is strictly concave on the observed face, strictly decreasing in the normal direction from it to the unobserved face and, otherwise, constant. For more details of the geometry of the observed face see the paper [3].…”

“…A working model M can be represented by a subset of ∆ k and may be specified by an explicit parameterisation. Computational information geometry explicitly uses the information geometry of ∆ k to numerically compute statistically important features of M. These features include: properties of the likelihood, which can be nontrivial in many of the examples considered here; the adequacy of first order asymptotic methods -notably, via higher order asymptotic expansions; curvature based dimension reduction; and inference in mixture models, [3].…”