A Notion of Entropy for Stochastic Processes on Marked Rooted Graphs

Abstract

In this document, we introduce a notion of entropy for stochastic processes on marked rooted graphs. For this, we employ the framework of local weak limit theory for sparse marked graphs, also known as the objective method, due to Benjamini, Schramm, Aldous, Steele and Lyons [BS01, AS04, AL07]. Our contribution is a generalization of the notion of entropy introduced by Bordenave and Caputo [BC15] to graphs which carry marks on their vertices and edges.The theory of time series is the engine driving an enormous range of applications in areas such as control theory, communications, information theory and signal processing. It is to be expected that a theory of stationary stochastic processes indexed by combinatorial structures, in particular graphs, would eventually have a similarly wide-ranging impact. * ⊂ Ḡ * consist of isomorphism classes of rooted marked graphs where all the vertices of the connected component of the root are at distance at most h from the root. For instance, forFor a Polish space X, we say that a sequence of Borel probability measures (µ n ∈ P(X) : n ∈ N) converges weakly to µ ∈ P(X), and write µ n ⇒ µ, if, for any bounded continuous function f : X → R, we have f dµ n → f dµ. If X is Polish, then weak * to have the same mark component as g, i.e. g k [m] := g[m], and subgraph component the truncation of the subgraph component of g up to depth k, i.e. g k [s] := (g[s]) k . For a marked graph G, two adjacent vertices u, v in G, and h ≥ 1, we define the depth h type of the edge (u, v) as).(4)Note that we have employed the convention that the first component on the right hand * , we define, where (G, o) is an arbitrary member of [G, o]. This notation is well-defined, since E h (g, g ′ )(G, o), thought of as a function of (G, o) for fixed integer h ≥ 1 and g, g ′ ∈ Ξ × Ḡh−1 *, is invariant under rooted isomorphism.For h ≥ 1, P ∈ P( Ḡh * ), and g, g ′ ∈ Ξ × Ḡh−1 *, define e P (g, g ′ ) :Here, (G, o) is a member of the isomorphism class [G, o] that has law P . This notation is well-defined for the same reason as above.< ∞ and e P (g, g ′ ) = e P (g ′ , g) for all g, g ′ ∈ Ξ × Ḡh−1 * .The following simple lemma indicates the importance of the concept of admissibility.Lemma 1. Let h ≥ 1, and let µ ∈ P u ( Ḡ * ) be a unimodular probability measure with deg(µ) < ∞. Let P := µ h . Then P is admissible.

A Universal Lossless Compression Method applicable to Sparse Graphs and Heavy-Tailed Sparse Graphs

“…We call this notion the BC entropy. The authors of this paper have generalized this entropy to the regime where the vertices and edges in the graph also carry marks, but we omit that discussion here since we focus on unmarked graphs throughout this work [DA19a].…”

“…For instance, it can be shown that the BC entropy of a probability distribution µ ∈ P u (T * ) can be approximated in terms of the finite depth truncation of µ [BC15, Theorem 1.3]. The reader is also referred to [DA19a] for the generalization of this notion to the marked regime.…”

“…This was done by employing the notion of "local weak convergence", an instance of the so-called the "objective method" [BS01, AS04, AL07], which, roughly speaking, allows one to think of the graphical data as a sample from a limiting stochastic object derived from the empirical characteristics of the given sample (more precisely, this is done in an asymptotic setting, and the limiting stochastic object is a probability distribution on rooted graphs; details are given in Section 2.1). Moreover, the authors have built upon the work of Bordenave and Caputo [BC15] to introduce a notion of entropy called the marked BC entropy which turns out to be the correct information-theoretic measure of optimality on a per-edge basis (which is the same as the per-vertex basis in this sparsity regime) for the purpose of the universal compression of graphical data in this formulation of the compression problem [DA19a]. Note that compression to the correct information-theoretic limit on a per-edge basis is a significantly deeper guarantee of informationtheoretic optimality than a crude guarantee of matching the growth rate of the overall entropy of the graphical data, since the leading term in the overall entropy depends only on the average degree of the graph and is on the scale of n log n where n denotes the number of vertices; see the details in Section 2.…”

Distributed Compression of Graphical Data

“…We employ the framework of local weak convergence, also called the objective method, as a counterpart for marked graphs of the notion of stochastic processes for time series [8]- [10]. Our characterization of the rate region is best understood in terms of a generalization of a measure of entropy introduced by Bordenave and Caputo [11], which we call the marked BC entropy [12]. It turns out that, for the sequences of ensembles we study in this paper and even more generally, as proved in [12], this notion of entropy captures the per-vertex growth rate of the portion of the Shannon entropy of the graphical data that is over and above an entropy of connectivity which is controlled entirely by the average degree of the graph ensemble and not the detailed statistics of the graphical data.…”

“…Our characterization of the rate region is best understood in terms of a generalization of a measure of entropy introduced by Bordenave and Caputo [11], which we call the marked BC entropy [12]. It turns out that, for the sequences of ensembles we study in this paper and even more generally, as proved in [12], this notion of entropy captures the per-vertex growth rate of the portion of the Shannon entropy of the graphical data that is over and above an entropy of connectivity which is controlled entirely by the average degree of the graph ensemble and not the detailed statistics of the graphical data. Indeed, to the highest order, the marked BC entropy captures the part of the overall entropy that truly depends on the empirical characteristics of the graphical data and not just on the underlying connectivity structure of the graph.…”