Abstract:We propose new measures of shared information, unique information and synergistic information that can be used to decompose the mutual information of a pair of random variables (Y, Z) with a third random variable X. Our measures are motivated by an operational idea of unique information, which suggests that shared information and unique information should depend only on the marginal distributions of the pairs (X, Y ) and (X, Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures.

We introduce binomial edge ideals attached to a simple graph G and study their algebraic properties. We characterize those graphs for which the quadratic generators form a Gröbner basis in a lexicographic order induced by a vertex labeling. Such graphs are chordal and claw-free. We give a reduced squarefree Gröbner basis for general G. It follows that all binomial edge ideals are radical ideals. Their minimal primes can be characterized by particular subsets of the vertices of G. We provide sufficient conditions for Cohen-Macaulayness for closed and nonclosed graphs.Binomial edge ideals arise naturally in the study of conditional independence ideals. Our results apply for the class of conditional independence ideals where a fixed binary variable is independent of a collection of other variables, given the remaining ones. In this case the primary decomposition has a natural statistical interpretation.

How can the information that a set {X1, . . . , Xn} of random variables contains about another random variable S be decomposed? To what extent do different subgroups provide the same, i.e. shared or redundant, information, carry unique information or interact for the emergence of synergistic information? Recently Williams and Beer proposed such a decomposition based on natural properties for shared information. While these properties fix the structure of the decomposition, they do not uniquely specify the values of the different terms. Therefore, we investigate additional properties such as strong symmetry and left monotonicity. We find that strong symmetry is incompatible with the properties proposed by Williams and Beer. Although left monotonicity is a very natural property for an information measure it is not fulfilled by any of the proposed measures. We also study a geometric framework for information decompositions and ask whether it is possible to represent shared information by a family of posterior distributions. Finally, we draw connections to the notions of shared knowledge and common knowledge in game theory. While many people believe that independent variables cannot share information, we show that in game theory independent agents can have shared knowledge, but not common knowledge. We conclude that intuition and heuristic arguments do not suffice when arguing about information.

The information that two random variables Y , Z contain about a third random variable X can have aspects of shared information (contained in both Y and Z), of complementary information (only available from (Y, Z) together) and of unique information (contained exclusively in either Y or Z).Here, we study measures SI of shared, U I unique and CI complementary information introduced by Bertschinger et al. [1] which are motivated from a decision theoretic perspective. We find that in most cases the intuitive rule that more variables contain more information applies, with the exception that SI and CI information are not monotone in the target variable X. Additionally, we show that it is not possible to extend the bivariate information decomposition into SI, U I and CI to a non-negative decomposition on the partial information lattice of Williams and Beer [2]. Nevertheless, the quantities U I, SI and CI have a well-defined interpretation, even in the multivariate setting.