Information geometry and sufficient statistics

Abstract

Abstract. Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari-Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. This leads to the question how the geometric structures behave under such sufficient statistics. While this is well studied in the finite sample size case, in the infinite case, we encounter technical problems concerning the appropriate topologies. Here, we introduce notions of parametrized measure models and tensor fields on them that exhibit the right behavior under statistical transformations. Within this framework, we can then handle the topological issues and show that the Fisher metric and the AmariChentsov tensor on statistical models in the class of symmetric 2-tensor fields and 3-tensor fields can be uniquely (up to a constant) characterized by their invariance under sufficient statistics, thereby achieving a full generalization of the original result of Chentsov to infinite sample sizes. More generally, we decompose Markov morphisms between statistical models in terms of statistics. In particular, a monotonicity result for the Fisher information naturally follows. MSC2010: 53C99, 62B05

The uniqueness of the Fisher metric as information metric

“…We obtain immediately from the Ay-Jost-Lê-Schwachhöfer theorem [4,Remark 3.23] and Theorem 1.2 the following Corollary 1.3. Let {Ω} be the class of topological spaces provided with Borel σ-algebra.…”

“…An indeterministic decision rule is a Markov transition distribution [11]. Recently, Ay-Jost-Lê-Schwachhöfer showed that a transformation between statistical models which is associated with a Markov transition distribution is a composition of the inverse of a transformation, which is associated with a sufficient statistic, and a transformation which is associated with a statistic [4,Theorem 4.10]. Hence, assuming the condition of invariance under sufficient statistics, the "information loss" condition is reduced to the case where data processing is associated with a statistic.…”

“…The condition (2) has been considered in [4] as a criterion for a natural metric on parametrized measure models. The condition (1) is simpler formulated than the condition (2), since it does not depend on the notion of a sufficient statistic, that depends on a statistical model under consideration and depends on the notion of information implicitly.…”

“…In [4,Corollary 4.11], for finite sample spaces Ω, we derived the uniqueness (up to a multiplicative constant) of a metric that satisfies the condition (1) on statistical models associated with Ω from the uniqueness of a metric on statistical models that satisfies the condition (2) on Ω, see Proposition 6.2 and the Appendix at the end of this note for a discussion on the Chentsov theorem. The converse statement, every metric that satisfies the condition (2) also satisfies the condition (1), follows from the monotonicity theorem for the Fisher metric on statistical models associated with finite sample spaces.…”

“…This fact has been observed earlier for parametrized measure models associated with finite sample spaces by Campbell in [9]). Further, Theorem 3.11 in [4] states that, the Fisher metric satisfies (1) if Ω, Ω 1 are smooth manifolds and µ is dominated by a Lebesgue measure.…”