The Weak Aggregating Algorithm and Weak Mixability

Kalnishkan, Yuri; Vyugin, Michael V.

doi:10.1007/11503415_13

Cited by 10 publications

(14 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remark 2 In fact, the second difficulty is more apparent than real: for example, in the binary case (Y = {0, 1}) with the loss function λ(γ, y) independent of x, there are many non-trivial continuous prediction rules in the canonical form of the prediction game [45] with the prediction set redefined as the boundary of the set of superpredictions [19].…”

Section: Universal Consistency For Randomized Prediction Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Predictions as Statements and Decisions

Vovk

2006

Learning Theory

View full text Add to dashboard Cite

Prediction is a complex notion, and different predictors (such as people, computer programs, and probabilistic theories) can pursue very different goals. In this paper I will review some popular kinds of prediction and argue that the theory of competitive on-line learning can benefit from the kinds of prediction that are now foreign to it.The standard goal for predictor in learning theory is to incur a small loss for a given loss function measuring the discrepancy between the predictions and the actual outcomes. Competitive on-line learning concentrates on a "relative" version of this goal: the predictor is to perform almost as well as the best strategies in a given benchmark class of prediction strategies. Such predictions can be interpreted as decisions made by a "small" decision maker (i.e., one whose decisions do not affect the future outcomes).Predictions, or probability forecasts, considered in the foundations of probability are statements rather than decisions; the loss function is replaced by a procedure for testing the forecasts. The two main approaches to the foundations of probability are measure-theoretic (as formulated by Kolmogorov) and game-theoretic (as developed by von Mises and Ville); the former is now dominant in mathematical probability theory, but the latter appears to be better adapted for uses in learning theory discussed in this paper.An important achievement of Kolmogorov's school of the foundations of probability was construction of a universal testing procedure and realization (Levin, 1976) that there exists a forecasting strategy that produces ideal forecasts. Levin's ideal forecasting strategy, however, is not computable. Its more practical versions can be obtained from the results of game-theoretic probability theory. For a wide class of forecasting protocols, it can be shown that for any computable game-theoretic law of probability there exists a computable forecasting strategy that produces ideal forecasts, as far as this law of probability is concerned. Choosing suitable laws of probability we can ensure that the forecasts agree with reality in requisite ways.

show abstract

Section: Universal Consistency For Randomized Prediction Algorithmsmentioning

confidence: 99%

“…The second addend on the right-hand side of (19) tends to zero by the continuity of the mapping Q ∈ P(Y) → Y f (y)Q(dy) for a continuous f ( [7], III.4.2, Proposition 6).…”

Section: Remark 4 Another Popular Notion Of the Integral For Vector-vmentioning

confidence: 99%

Predictions as Statements and Decisions

Vovk

2006

Learning Theory

View full text Add to dashboard Cite

show abstract

“…An especially important class of loss functions is that of "mixable" ones, for which the learner's loss can be made as small as the best expert's loss plus a constant (depending on the number of experts). It is known (Haussler et al, 1998;Vovk, 1998) that the optimal additive constant is attained by the "strong aggregating algorithm" proposed in Vovk (1990) (we use the adjective "strong" to distinguish it from the "weak aggregating algorithm" of Kalnishkan & Vyugin, 2005). …”

Section: Introductionmentioning

confidence: 99%

Prediction with expert advice for the Brier game

Vovk

Zhdanov

2008

Proceedings of the 25th International Conference on Machine Learning - ICML '08

View full text Add to dashboard Cite

show abstract

“…This concept has long 2 In [KV08] and other earlier papers it was required that for every γ0 ∈ Γ such that λ(ω * , γ0) = +∞ for some ω * ∈ Ω there should be a sequence γ1, γ2, . .…”

Section: Generalised Entropiesmentioning

confidence: 99%

“…In order to get an effective version of Proposition 1, one needs to restate results of [KV08] in an effective fashion. The procedures used in [KV08] are essentially effective (and efficient) but require certain properties of Γ and λ; otherwise the prediction space and the loss function can be distorted in such a way as to make the procedures from [KV08] unusable. Formalising these properties in a simple form appears to be a difficult task.…”

Section: Computable Gamesmentioning

confidence: 99%

Generalised Entropy and Asymptotic Complexities of Languages

Self Cite

View full text Add to dashboard Cite

The paper explores connections between asymptotic complexity and generalised entropy. Asymptotic complexity of a language (a language is a set of finite or infinite strings) is a way of formalising the complexity of predicting the next element in a sequence: it is the loss per element of a strategy asymptotically optimal for that language. Generalised entropy extends Shannon entropy to arbitrary loss functions; it is the optimal expected loss given a distribution on possible outcomes. It turns out that the set of tuples of asymptotic complexities of a language w.r.t. different loss functions can be described by means of the generalised entropies corresponding to the loss functions.

show abstract

The Weak Aggregating Algorithm and Weak Mixability

Cited by 10 publications

References 4 publications

Predictions as Statements and Decisions

Predictions as Statements and Decisions

Prediction with expert advice for the Brier game

Generalised Entropy and Asymptotic Complexities of Languages

Contact Info

Product

Resources

About