Compression of Graphical Structures: Fundamental Limits, Algorithms, and Experiments

Abstract

Information theory traditionally deals with "conventional data," be it textual data, image, or video data. However, databases of various sorts have come into existence in recent years for storing "unconventional data" including biological data, social data, web data, topographical maps, and medical data. In compressing such data, one must consider two types of information: the information conveyed by the structure itself, and the information conveyed by the data labels implanted in the structure. In this paper, we attempt to address the former problem by studying information of graphical structures (i.e., unlabeled graphs). As the first step, we consider the Erdős-Rényi graphs G(n, p) over n vertices in which edges are added randomly with probability p. We prove that the structural entropy of G(n, p) iswhere h(p) = −p log p − (1 − p) log(1 − p) is the entropy rate of a conventional memoryless binary source. Then, we propose a two-stage compression algorithm that asymptotically achieves the structural entropy up to the first two leading terms. Our algorithm runs in O(n+e) time on average where e is the number of edges. To the best of our knowledge, this is the first provable (asymptotically) optimal graph compressor. We use combinatorial and analytic techniques such as generating functions, Mellin transform, and poissonization to establish these findings. Our experiments confirm theoretical results and show the usefulness of our algorithm for real-world graphs such as the Internet, biological networks, and social networks.

Learning of Graph Compressed Dictionaries for Sparse Representation Classification

“…Therefore, it can be easily convert data to the adjacency matrix of a weighted graph. Compression methods can be addressed with an Information-theoretic approach by compressing graphical structures (Choi and Szpankowski, 2012) without preventing a graph structure as the compressed representation. As a second category can be presented as regularity pairs by Szemerédi regularity lemma (Szemerédi, 1978) that is a well-known result in extremal graph theory, which roughly states that a dense graph can be approximated by a bounded number of random bipartite graphs.…”

“…One way to reduce memory and time complexity, and increase accuracy is data compression (Choi and Szpankowski, 2012;Navlakha et al, 2008;Toivonen et al, 2011) as applied in big data applications (Nourbakhsh, 2015). Compressing data consists in changing its representation in a way that requires fewer bits.…”

Capacity of a Structural Binary Symmetric Channel

“…This opens unbounded opportunities for information theory to extend its scope beyond its original goals, that of communication and storage. We suggest [3], [10] to broaden information theory to study finite size data structures (e.g., graphs, sets, social networks), that is, to develop information theory of data structures beyond first-order asymptotics. In Fig.…”

“…1. Effect of BSC Channel on Graphs particular, in [3] as the first step in understanding structural information, we explore structures on graphs, specifically, we study unlabeled graphs (or structures) and defined structural entropy characterizing graph compression.…”

“…Finally, in a recent paper Kieffer et al [6] presented a structural complexity of a binary tree. The structural entropy was introduced in [3] where the first provable (asymptotically) optimal graph compressor for Erdős-Rényi graph models was presented. To the best of our knowledge structural binary symmetric channels were not discussed in open literature.…”