Kolmogorov Complexity and its Applications

Li, Ming; Vitányi, Paul M. B.

doi:10.1016/b978-0-444-88071-0.50009-6

Cited by 79 publications

(58 citation statements)

References 126 publications

(168 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although inherently incomputable, -complexity can be easily approximated to a degree which allows for the practical use of -complexity in real-world applications, for instance, in machine learning [26,27], computer network management [28], and general computation theory (proving lower bounds of various Turing machines, combinatorics, formal languages, and inductive inference) [29].…”

Section: -Complexity As the Measure Of Network Complexitymentioning

confidence: 99%

On Measuring the Complexity of Networks: Kolmogorov Complexity versus Entropy

2017

View full text Add to dashboard Cite

One of the most popular methods of estimating the complexity of networks is to measure the entropy of network invariants, such as adjacency matrices or degree sequences. Unfortunately, entropy and all entropy-based information-theoretic measures have several vulnerabilities. These measures neither are independent of a particular representation of the network nor can capture the properties of the generative process, which produces the network. Instead, we advocate the use of the algorithmic entropy as the basis for complexity definition for networks. Algorithmic entropy (also known as Kolmogorov complexity or -complexity for short) evaluates the complexity of the description required for a lossless recreation of the network. This measure is not affected by a particular choice of network features and it does not depend on the method of network representation. We perform experiments on Shannon entropy and -complexity for gradually evolving networks. The results of these experiments point to -complexity as the more robust and reliable measure of network complexity. The original contribution of the paper includes the introduction of several new entropy-deceiving networks and the empirical comparison of entropy and -complexity as fundamental quantities for constructing complexity measures for networks.

show abstract

Section: -Complexity As the Measure Of Network Complexitymentioning

confidence: 99%

On Measuring the Complexity of Networks: Kolmogorov Complexity versus Entropy

2017

View full text Add to dashboard Cite

show abstract

“…Kemeny suggests adopting a tvle of inference, which is to select the simplest hypothesis compatible wi[h [he observed data. This is his`Rule 3' (Kemeny, 1953, p. 397), which has been named`Kemeny's Rule' hy Li and Vitányi (1992). Althuugh intuitivcly appcaling, this is rathcr ud hoc.…”

Section: Lurv R~1'rucinrorn~(ur 1'urc Imonv)mentioning

confidence: 99%

“…Sulomonoff givrs intuitivc as well as yuasi-furmal mulivalions fnr thix assignment (a purely formal motivation can he t2~und in Li and Vitányi, 1992). The intuitive motivation is based on two arguments.…”

Section: And Levin ( See Rissanen 19r3)mentioning

confidence: 99%

“…In case of (3), it can be shown that 2~K~s"~~Pc,-r-tic(Su) (e.g. Li and Vitányi, 1992). It turns out that Jeffreys' prior probability dislribution, based on [he Jeffreys-Wrinch Simplicity Postulate, is an approxirnation to the ideal (or 'universal', as Rissanen, 1983 calls it) prior prohability distrihution.l(~The Solomonuff-t,e:vin distríbutiun is not based on subjective Bayesian hctting, hut on 'objective' fcatures of the problem of interest.…”

Section: And Levin ( See Rissanen 19r3)mentioning

confidence: 99%

“…Li and Vitányi, 1992). Roughly speaking, K measures the minimum number of hits required to encode a proposition.…”

Section: It Can He Shown That Lo4p(h;) Is Related To the Descriptive mentioning

confidence: 99%

See 2 more Smart Citations

Simplicity, Scientific Inference and Econometric Modelling

Keuzenkamp

McAleer

1995

The Economic Journal

View full text Add to dashboard Cite

Protocols for Asymmetric Communication Channels

Adler

Maggs

2001

Journal of Computer and System Sciences

View full text Add to dashboard Cite

In this paper we examine the problem of sending an n-bit data item from a client to a server across an asymmetric communication channel. We demonstrate that there are scenarios in which a high-speed link from the server to the client can be used to greatly reduce the number of bits sent from the client to the server across a slower link. In particular, we assume that the data item is drawn from a probability distribution D that is known to the server but not to the client. We present several protocols in which the expected number of bits transmitted by the server and client are O(n) and O(H(D)+1), respectively, where H(D) is the binary entropy of D (and can range from 0 to n). These protocols are within a small constant factor of optimal in terms of the number of bits sent by the client. The expected number of rounds of communication between the server and client in the simplest of our protocols is O (H(D)). We also give a protocol for which the expected number of rounds is only O(1), but which requires more computational effort on the part of the server. A third technique provides a tradeoff between the computational effort and the number of rounds. These protocols are complemented by several lower bounds and impossibility results. We prove that all of our protocols are existentially optimal in terms of the number of bits sent by the server, i.e., there are distributions for which the total number of bits exchanged has to be at least n. In addition, we show that there is no protocol that is optimal for every distribution (as opposed to just existentially optimal) in terms of bits sent by the server. We demonstrate this by proving that it is undecidable to compute (even approximately), for an arbitrary distribution D, the expected number of bits that must be exchanged by the server and client on the distribution D.

show abstract

Kolmogorov Complexity and its Applications

Cited by 79 publications

References 126 publications

On Measuring the Complexity of Networks: Kolmogorov Complexity versus Entropy

On Measuring the Complexity of Networks: Kolmogorov Complexity versus Entropy

Simplicity, Scientific Inference and Econometric Modelling

Protocols for Asymmetric Communication Channels

Contact Info

Product

Resources

About