Short Lists with Short Programs in Short Time

Bauwens, Bruno; Makhlin, Anton; Vereshchagin, Nikolai K.; Zimand, Marius

doi:10.1109/ccc.2013.19

Cited by 26 publications

(69 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the source should be computable from the code. 7 A dual topic is what is known as randomness or dimension extraction which, roughly speaking, asks for the effective transformation of a given stream X into a stream Y which is algorithmically random or, at least, has higher Hausdorff dimension than X. Although this problem is only tangential to our topic, it is very related to the work of Ryabko [35,36] and later Doty [14] which we discuss in the following.…”

mentioning

confidence: 99%

Compression of Data Streams Down to Their Information Content

Barmpalias

Lewis-Pye

2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

According to Kolmogorov complexity, every finite binary string is compressible to a shortest code -its information content -from which it is effectively recoverable. We investigate the extent to which this holds for infinite binary sequences (streams). We devise a new coding method which uniformly codes every stream X into an algorithmically random stream Y, in such a way that the first n bits of X are recoverable from the first I(X ↾ n ) bits of Y, where I is any partial computable information content measure which is defined on all prefixes of X, and where X ↾ n is the initial segment of X of length n. As a consequence, if g is any computable upper bound on the initial segment prefix-free complexity of X, then X is computable from an algorithmically random Y with oracle-use at most g. Alternatively (making no use of such a computable bound g) one can achieve an oracle-use bounded above by K(X ↾ n ) + log n. This provides a strong analogue of Shannon's source coding theorem for algorithmic information theory.A fruitful way to quantify the complexity of a finite object such as a string σ in a finite alphabet 1 is to consider the length of the shortest binary program which prints σ. This fundamental idea gives rise to a theory of algorithmic information and compression, which is based on the theory of computation and was pioneered by Kolmogorov [21] and Solomonoff [41]. The Kolmogorov complexity of a binary string σ is the length of the shortest program that outputs σ with respect to a fixed universal Turing machine. The use of prefix-free machines in the definition of Kolmogorov complexity was pioneered by Levin [26] and Chaitin [10], and allowed for the development of a robust theory of incompressibility and algorithmic randomness for streams (i.e. infinite binary sequences). Information content measures, defined by Chaitin [11] after Levin [26], are functions that assign a positive integer value to each binary string, representing the amount of information contained in the string.Definition 1.1 (Information content measure). A partial function I from strings to N is an information content measure if it is right-c.e. 2 and I(σ)↓ 2 −I(σ) is finite.Prefix-free Kolmogorov complexity can be characterized as the minimum (modulo an additive constant) information content measure. If K(σ) denotes the prefix-free Kolmogorov complexity of the string σ, then for c ∈ N we say that σ is c-incompressible if K(σ) ≥ |σ| − c. It is a basic fact concerning Kolmogorov complexity that for some universal constant c: every string σ has a shortest code σ * which is itself c-incompressible.(1)Our goal is to investigate the extent to which the above fact holds in an infinite setting, i.e. for streams instead of strings. In the context of Kolmogorov complexity, algorithmic randomness is defined as incompressibility. 3 So (1) can be read as follows: we can uniformly code each string σ into an algorithmically random string of length K(σ). In order to formalise an infinitary analogue of this statement, we need to make use of oracle-machine ...

show abstract

mentioning

confidence: 99%

Compression of Data Streams Down to Their Information Content

Barmpalias

Lewis-Pye

2019

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…In this case ϕ p is understood as ϕ(p) if defined and as a special symbol ⊥ otherwise. Optimal Gödel numberings for m = 0 were called standard machines in [1] and we will use the same terminology. Kolmogorov complexity C U (x) of a string x with respect to a standard machine U is the usual Kolmogorov complexity (the minimal length of a U -program for x).…”

Section: Resultsmentioning

confidence: 99%

“…The paper [1] shows that for every standard machine U , given a string x we can find a short list of strings with a short program for x: the size (=cardinality) of the list is O(|x| 2 ) and it contains a U -program for x of length at most C U (x) + O (1).…”

Section: Resultsmentioning

confidence: 99%

“…This question was asked recently by Alexander Shen [5]. Notice that there is no total algorithm that maps any program for x to x (otherwise the positive answer to the question would immediately follow from the cited result of [1]). We show that for every standard machine U and for every function ε of p there are infinitely pairs (x, its U -program p) such that the size of L(p) is exponential in both |x| − ε and |p| − ε provided L(p) has a program for x of length at most C U (x) + ε.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Short lists with short programs from programs of functions and strings

Vereshchagin

2017

Theory Comput Syst

Self Cite

View full text Add to dashboard Cite

Let {ϕ p } be an optimal Gödel numbering of the family of computable functions (in Schnorr's sense), where p ranges over binary strings. Assume that a list of strings L(p) is computable from p and for all p contains a ϕ-program for ϕ p whose length is at most ε bits larger that the length of the shortest ϕ-program for ϕ p . We show that for infinitely many p the list L(p) must have 2 |p|−ε−O(1) strings. Here ε is an arbitrary function of p. * The work was done while visiting IMS (University of Singapore), the program "Algorithmic Randomness", 2-30 June 2014. The work was in part supported by the RFBR grant 12-01-00864.

show abstract

“…The trade off between C(A) and log |A| is represented by the profile of x. 1 Kolmogorov complexity of A is defined as follows. We fix any computable bijection A → [A] from the family of all finite sets to the set of binary strings, called encoding.…”

Section: Algorithmic Statistics: Basic Notionsmentioning

confidence: 99%

Algorithmic Statistics: Normal Objects and Universal Models

Milovanov

2016

Computer Science – Theory and Applications

View full text Add to dashboard Cite

Kolmogorov suggested to measure quality of a statistical hypothesis (a model) P for a data x by two parameters: Kolmogorov complexity C(P ) of the hypothesis and the probability P (x) of x with respect to P . The first parameter measures how simple the hypothesis P is and the second one how it fits. The paper [2] discovered a small class of models that are universal in the following sense. Each hypothesis Sij from that class is identified by two integer parameters i, j and for every data x and for each complexity level α there is a hypothesis Sij with j i l(x) of complexity at most α that has almost the best fit among all hypotheses of complexity at most α. The hypothesis Sij is identified by i and the leading i − j bits of the binary representation of the number of strings of complexity at most i. On the other hand, the initial data x might be completely irrelevant to the the number of strings of complexity at most i. Thus Sij seems to have some information irrelevant to the data, which undermines Kolmogorov's approach: the best hypotheses should not have irrelevant information.To restrict the class of hypotheses for a data x to those that have only relevant information, the paper [10] introduced a notion of a strong model for x: those are models for x whose total conditional complexity conditional to x is negligible. An object x is called normal if for each complexity level α at least one its best fitting model of that complexity is strong.In this paper we show that there are "many types" of normal strings (Theorem 10). Our second result states that there is a normal object x such that all its best fitting models Sij are not strong for x. Our last result states that every best fit strong model for a normal object is again a normal object.

show abstract

Short Lists with Short Programs in Short Time

Cited by 26 publications

References 31 publications

Compression of Data Streams Down to Their Information Content

Compression of Data Streams Down to Their Information Content

Short lists with short programs from programs of functions and strings

Algorithmic Statistics: Normal Objects and Universal Models

Contact Info

Product

Resources

About