Algorithmic complexity bounds on future prediction errors

Abstract

We bound the future loss when predicting any (computably) stochastic sequence online. Solomonoff finitely bounded the total deviation of his universal predictor M from the true distribution μ by the algorithmic complexity of μ . Here we assume that we are at a time t>1 and have already observed x = x 1...xt . We bound the future prediction performance on xt+1xt+2... by a new variant of algorithmic complexity of μ given x, plus the complexity of the randomness deficiency of x. The new complexity is monotone in its condition in the sense that this complexity can only decrease if the condition is prolonged. We also briefly discuss potential generalizations to Bayesian model classes and to classification problems

Kolmogorov Complexity and Algorithmic Randomness

“…Если рассматривать условия как вершины дерева и требовать монотонности по этому аргументу, получатся четыре других вида условной сложности, которые, од-нако, практически не рассматривались (работа [29] | одно из редких исключений).…”

Plain stopping time and conditional complexities revisited

“…This notion was introduced by Vovk and Pavlovic [8]. It turns out that plain stopping time complexity of a binary string x could be equivalently defined as (a) the minimal plain complexity of a Turing machine that stops after reading x on a one-directional input tape; (b) the minimal plain complexity of an algorithm that enumerates a prefix-free set containing x; (c) the conditional complexity C(x | x * ) where x in the condition is understood as a prefix of an infinite binary sequence while the first x is understood as a terminated binary string; (d) as a minimal upper semicomputable function K such that each binary sequence has at most 2 n prefixes z such that K(z) < n; (e) as max C X (x) where C X (z) is plain Kolmogorov complexity of z relative to oracle X and the minimum is taken over all extensions X of x.We also show that some of these equivalent definitions become non-equivalent in the more general setting where the condition y and the object x may differ, and answer some open question from Chernov, Hutter and Schmidhuber [2].…”

“…We also show that some of these equivalent definitions become non-equivalent in the more general setting where the condition y and the object x may differ, and answer some open question from Chernov, Hutter and Schmidhuber [2].…”

“…Unfortunately, here the situation is much worse, and we prove mostly negative results (Section 3). We show that while C(y | x * ) can be defined as a minimal plain complexity of a prefix-stable program that maps x to y (Theorem 7), it cannot be defined as a minimal plain complexity of a prefix-free program that maps some prefix of x to y (Theorem 8; this result answers a question posed in [2]). Then we show that the attempt to define C(y | x * ) by quantitative restrictions also fails: we get another function that may be up to two times less (Theorem 9).…”

“…The optimal set U can be constructed as If we omit the third requirement for description modes, we get the standard conditional complexity C(y | x) in the same way. The notation we use (placing * after the condition) follows [2] though a different version of monotone-conditional complexity is considered there. In general, C(y | x * ) is greater than the standard conditional complexity C(y | x) since we have more requirements for the description modes.…”