Symmetries among Multivariate Information Measures Explored Using Möbius Operators

Abstract

Relations between common information measures include the duality relations based on Möbius inversion on lattices, which are the direct consequence of the symmetries of the lattices of the sets of variables (subsets ordered by inclusion). In this paper we use the lattice and functional symmetries to provide a unifying formalism that reveals some new relations and systematizes the symmetries of the information functions. To our knowledge, this is the first systematic examination of the full range of relationships of this class of functions. We define operators on functions on these lattices based on the Möbius inversions that map functions into one another, which we call Möbius operators, and show that they form a simple group isomorphic to the symmetric group S3. Relations among the set of functions on the lattice are transparently expressed in terms of the operator algebra, and, when applied to the information measures, can be used to derive a wide range of relationships among diverse information measures. The Möbius operator algebra is then naturally generalized which yields an even wider range of new relationships.

Understanding the Evolution of Tree Size Diversity within the Multivariate Nonsymmetrical Diffusion Process and Information Measures

“…. , M, are defined by Equations (9)-(12) after plugging the mean and variance given by Equations (20) and (21).…”

“…Mutual information, I, between two random variables, X s and X u , compares the uncertainty of measuring variables jointly with the uncertainty of measuring the two variables independently, identifies nonlinear dependence between two variables [41][42][43], and is non-negative and symmetrical. A generalization of bivariate mutual information to more than two variables have been analyzed in few different scenarios [20,21,[41][42][43]. A direct multivariate extension of bivariate mutual information expressed by Equation 45to n variables X 1 , X 2 , and X n is named as the multi-information [44,45], also known as total correlation, and is defined by:…”

“…Generalized forms of mutual information to more than two variables are called interaction information [21,47]. The relationship between multi-information and interaction information for the trivariate and 4-variate cases are defined in the following forms [21]:…”

“…The greatest advantage of multivariate SDE approach is that it provides sufficient flexibility to fit a large variety of nested models for a separate tree size and stand size variable, which facilitates the selection and comparison of newly developed models by using information measures technique [19][20][21]. In order to construct the multi-information measures, it is necessary to obtain the probability density function of tree size variable, which in this study are obtained from a 4-variate Bertalanffy SDE describing the development of the tree size variables against the age.…”

Towards an information theory of quantitative genetics

“…The symmetries of the relationships among the information functionals are surprisingly simple, but also subtle. The multiple measures of information theory have strikingly symmetric relations [16,17], and have a number of symmetries that we have previously reported [18]. The symmetries all derive from the fact that all information measures are specific linear combinations of joint entropies, organized by lattices whose partial order is determined by inclusion of variable subsets.…”