Recent Progresses in Characterising Information Inequalities

Abstract

In this paper, we present a revision on some of the recent progresses made in characterising and understanding information inequalities, which are the fundamental physical laws in communications and compression. We will begin with the introduction of a geometric framework for information inequalities, followed by the first non-Shannon inequality proved by Zhang et al. in 1998 [1]. The discovery of this non-Shannon inequality is a breakthrough in the area and has led to the subsequent discovery of many more non-Shannon inequalities. We will also review the close relations between information inequalities and other research areas such as Kolmogorov complexity, determinantal inequalities, and group-theoretic inequalities. These relations have led to non-traditional techniques in proving information inequalities and at the same time made impacts back on those related areas by the introduction of information-theoretic tools.

Entropy Region and Convolution

“…The cone cl(H ent N ) is not polyhedral [35] and the structure of non-Shannon inequalities seems to be complex [45,28,13,14,11,43]. Reviews are in [9,34] and elsewhere.…”

“…Information-theoretic inequalities are under investigation in additive combinatorics [27]. Last but not least, the information-theoretic inequalities are known to be related to Kolmogorov complexity [28], determinantal inequalities and group-theoretic inequalities [9].…”

On a Connection between Information and Group Lattices

“…Seeking new information inequalities is currently an active research topic [27,36,39,40]; see Chan's review of recent progresses [41]. In fact, they should be more accurately called "laws of joint information", since these inequalities involves only joint information only.…”

Conditional Information Inequalities for Entropic and Almost Entropic Points

“…Linear inequalities for entropies are "general laws" of information which are widely used in information theory to describe fundamental limits in information transmission, compression, secrecy, etc. Information inequalities have also applications beyond information theory, e.g., in combinatorics and in group theory, see the survey in [5]. So, it is natural to ask whether there exist linear inequalities for entropy that hold for all distributions but are not Shannon-type.…”