On a Connection between Information and Group Lattices

Abstract

In this paper we review a particular connection between information theory and group theory. We formalize the notions of information elements and information lattices, first proposed by Shannon. Exploiting this formalization, we expose a comprehensive parallelism between information lattices and subgroup lattices. Qualitatively, isomorphisms between information lattices and subgroup lattices are demonstrated. Quantitatively, a decisive approximation relation between the entropy structures of information lattices and the log-index structures of the corresponding subgroup lattices, first discovered by Chan and Yeung, is highlighted. This approximation, addressing both joint and common entropies, extends the work of Chan and Yeung on joint entropy. A consequence of this approximation result is that any continuous law holds in general for the entropies of information elements if and only if the same law holds in general for the log-indices of subgroups. As an application, by constructing subgroup counterexamples, we find surprisingly that common information, unlike joint information, obeys neither the submodularity nor the supermodularity law. We emphasize that the notion of information elements is conceptually significant-formalizing it helps to reveal the deep connection between information theory and group theory. The parallelism established in this paper admits an appealing group-action explanation and provides useful insights into the intrinsic structure among information elements from a group-theoretic perspective.

Optical properties of periodic, quasi-periodic, and disordered one-dimensional photonic structures

“…' In' optical' and' condensed' matter' physics,' researchers' have' used' the' Shannon' entropy' variation' to' assess' the' stability' of' a' single' photon' based' quantum' cryptography' protocol' [216],'to'determine'the'probability'of'electronic'charge'distribution'between'the'atoms'of'a' benzene' ring' (also' named' aromaticity' measure)' [217],' and' to' evaluate' the' electron' localization'in'a'molecular'system' [218]. 'Moreover,'a'parallelism'in'condensed'matter'physics' among'information'lattice'and'subgroup'lattice'has'been'reported' [219].' In' two' dimensional' photonic' structures,' it' has' been' demonstrated' the' normalized' total' transmission'Tnt'can'be'related'to'the'Shannon'index'H''approximately'with'this'simple'linear' behaviour' (17)' ' that'shows'an'increase'of'the'normalized'total'transmission'by'increasing'the'Shannon'index' [220,221].…”

Intersection Information Based on Common Randomness

“…Given random variables X and Y , we write X Y to signify that there exists a measurable function, f , such that X = f (Y ) almost surely (i.e., with probability one). In this case, following the terminology in [8], we say that X is informationally poorer than Y ; this induces a partial order on the set of random variables. Similarly, we write X Y if Y X, in which case we say X is informationally richer than Y .…”

“…If X and Y are such that X Y and X Y , then we write X ∼ = Y . In this case, again following [8], we say that X and Y are informationally equivalent. In other words, X ∼ = Y if and only if one can relabel the values of X to obtain a random value that is equal to Y almost surely and vice versa.…”

“…Next, we follow [8] and consider the join and meet operators. These operators were defined for information elements, which are σ-algebras or, equivalently, equivalence classes of random variables.…”

Some New Insights Into Information Decomposition in Complex Systems Based on Common Information

“…Given an underlying probability space, there exists a natural one-to-one mapping between sample-space partitions and σ-algebras. This implies we can partition the set of all RVs into disjoint equivalence classes, called information elements, s.t., all RVs within a given class are informationally equivalent [17], [18]. Shannon then defined a relation of inclusion between two such information elements: we say that Y (X) is informationally richer…”