On the entropy region of discrete and continuous random variables and network information theory

Abstract

Abstract-We show that a large class of network information theory problems can be cast as convex optimization over the convex space of entropy vectors. A vector in 2 n − 1 dimensional space is called entropic if each of its entries can be regarded as the joint entropy of a particular subset of n random variables (note that any set of size n has 2 n − 1 nonempty subsets.) While an explicit characterization of the space of entropy vectors is wellknown for n = 2, 3 random variables, it is unknown for n > 3 (which is why most network information theory problems are open.) We will construct inner bounds to the space of entropic vectors using tools such as quasi-uniform distributions, lattices, and Cayley's hyperdeterminant.

A parallel dynamic programming approach for data analysis

“…One of the most commonly used probability distribution evaluation function is entropy. [6] The definition of information theory entropy [10] is defined as, [1] …”