Low-algorithmic-complexity entropy-deceiving graphs

Abstract

In estimating the complexity of objects, in particular, of graphs, it is common practice to rely on graph- and information-theoretic measures. Here, using integer sequences with properties such as Borel normality, we explain how these measures are not independent of the way in which an object, such as a graph, can be described or observed. From observations that can reconstruct the same graph and are therefore essentially translations of the same description, we see that when applying a computable measure such as the Shannon entropy, not only is it necessary to preselect a feature of interest where there is one, and to make an arbitrary selection where there is not, but also more general properties, such as the causal likelihood of a graph as a measure (opposed to randomness), can be largely misrepresented by computable measures such as the entropy and entropy rate. We introduce recursive and nonrecursive (uncomputable) graphs and graph constructions based on these integer sequences, whose different lossless descriptions have disparate entropy values, thereby enabling the study and exploration of a measure's range of applications and demonstrating the weaknesses of computable measures of complexity.

On Measuring the Complexity of Networks: Kolmogorov Complexity versus Entropy

“…In this section, we present four different examples of entropy-deceiving networks, similar to the idea coined in [24]. Each of these networks has a simple generative procedure and should not (intuitively) be treated as complex.…”

“…Zenil et al [24] argue that entropy is not appropriate to measure the true complexity of a network and they present several examples of networks which should not qualify as complex (using the colloquial understanding of the term), yet which attain maximum entropy of various network invariants. We follow the line of argumentation of Zenil et al, and we present more examples of entropy-deceiving networks.…”

“…Rather, we follow the observations formulated in [24] and we present the criticism of the entropy as the guiding principle of complexity measure construction. Thus, we do not use any specific formal definition of complexity, but we provide additional arguments why entropy may be easily deceived when trying to evaluate the complexity of a network.…”

Symmetry and Correspondence of Algorithmic Complexity over Geometric, Spatial and Topological Representations

“…As with π, a graph that is produced recursively enumerable will be eventually characterized by algorithmic probability as having low algorithmic complexity, unlike traditional compression algorithms that implement a version of classical block Shannon entropy. In previous work, this kind of recursively enumerable graph [5] has been featured, illustrating how inappropriate Shannon entropy can be when there is a need for a universal, unbiased measure where no feature has to be pre-selected.…”

“…However, the extent to which entropy can be used to characterize symmetry is limited to only apparent symmetry, and it is not robust in the face of object description, due to its dependence on probability distributions [5]. Entropy measures the uncertainty associated with a random variable, i.e., the expected value of the information in a message (e.g., a string) in bits.…”