Competitive On‐line Statistics

Vovk, Vladimir

doi:10.1111/j.1751-5823.2001.tb00457.x

Cited by 182 publications

(212 citation statements)

References 51 publications

Supporting

Mentioning

209

Contrasting

Order By: Relevance

“…As the name suggests, this class of algorithms uses the squared loss function, and possesses a proven relative loss bound under our label noise model (Vovk, 2001; Dekel et al, 2012), with the desired sublinear growth. Established results for the algorithm will be used to derive our query condition (Section 2.6).…”

Section: Methodsmentioning

confidence: 99%

“…State of the art methods in selective sampling, with theoretical performance guarantees, include BBQ (Orabona and Cesa-Bianchi, 2011) and DGS (Dekel et al, 2012). These methods also use variants of the RLS algorithm (Azoury and Warmuth, 2001; Vovk, 2001; Auer, 2003; Cesa-Bianchi et al, 2005; Cesa-Bianchi and Lugosi, 2006; Cavallanti et al, 2008; Strehl and Littman, 2008; Cesa-Bianchi et al, 2009), and maintain a data correlation matrix to calculate a confidence interval or uncertainty level in their prediction, which is essentially an estimate of the variance of the RLS margin for the current instance.…”

Section: Methodsmentioning

confidence: 99%

“…The general framework is formulated as a selective sampling problem in which an agent samples any action in its current situation as soon as it sees that the effects of this action are statistically unknown. We present one possible implementation of the framework, using online linear classifiers (Azoury and Warmuth, 2001; Vovk, 2001; Cesa-Bianchi and Lugosi, 2006) as predictive action models , which essentially predict some aspects of the next state, given the current state-action features.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots

et al. 2013

View full text Add to dashboard Cite

A reinforcement learning agent that autonomously explores its environment can utilize a curiosity drive to enable continual learning of skills, in the absence of any external rewards. We formulate curiosity-driven exploration, and eventual skill acquisition, as a selective sampling problem. Each environment setting provides the agent with a stream of instances. An instance is a sensory observation that, when queried, causes an outcome that the agent is trying to predict. After an instance is observed, a query condition, derived herein, tells whether its outcome is statistically known or unknown to the agent, based on the confidence interval of an online linear classifier. Upon encountering the first unknown instance, the agent “queries” the environment to observe the outcome, which is expected to improve its confidence in the corresponding predictor. If the environment is in a setting where all instances are known, the agent generates a plan of actions to reach a new setting, where an unknown instance is likely to be encountered. The desired setting is a self-generated goal, and the plan of action, essentially a program to solve a problem, is a skill. The success of the plan depends on the quality of the agent's predictors, which are improved as mentioned above. For validation, this method is applied to both a simulated and real Katana robot arm in its “blocks-world” environment. Results show that the proposed method generates sample-efficient curious exploration behavior, which exhibits developmental stages, continual learning, and skill acquisition, in an intrinsically-motivated playful agent.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots

et al. 2013

View full text Add to dashboard Cite

show abstract

“…There are good introductions and surveys of Solomonoff sequence prediction Vitányi, 1992, 1997), inductive inference in general (Angluin and Smith, 1983, Solomonoff, 1997, Merhav and Feder, 1998, reasoning under uncertainty (Grünwald, 1998), and competitive online statistics (Vovk, 1999), with interesting relations to this work. See Section 6.3 for some more details.…”

Section: Introductionmentioning

confidence: 99%

“…Early works in this direction are Dawid (1984), Rissanen (1989). See Vovk (1999) for a review and further references. We describe the setting and basic idea of PEA for binary alphabet.…”

mentioning

confidence: 99%

10.1162/1532443041827952

Hutter

2000

CrossRef Listing of Deleted DOIs

View full text Add to dashboard Cite

Various optimality properties of universal sequence predictors based on Bayes-mixtures in general, and Solomonoff's prediction scheme in particular, will be studied. The probability of observing x t at time t, given past observations x 1 ...x t−1 can be computed with the chain rule if the true generating distribution µ of the sequences x 1 x 2 x 3 ... is known. If µ is unknown, but known to belong to a countable or continuous class M one can base ones prediction on the Bayes-mixture ξ defined as a w ν -weighted sum or integral of distributions ν ∈ M. The cumulative expected loss of the Bayes-optimal universal prediction scheme based on ξ is shown to be close to the loss of the Bayes-optimal, but infeasible prediction scheme based on µ. We show that the bounds are tight and that no other predictor can lead to significantly smaller bounds. Furthermore, for various performance measures, we show Pareto-optimality of ξ and give an Occam's razor argument that the choice w ν ∼ 2 −K(ν) for the weights is optimal, where K(ν) is the length of the shortest program describing ν. The results are applied to games of chance, defined as a sequence of bets, observations, and rewards. The prediction schemes (and bounds) are compared to the popular predictors based on expert advice. Extensions to infinite alphabets, partial, delayed and probabilistic prediction, classification, and more active systems are briefly discussed.

show abstract

A hybrid statistical‐dynamical framework for meteorological drought prediction: Application to the southwestern United States

Madadgar

AghaKouchak

Shukla

et al. 2016

Water Resources Research

View full text Add to dashboard Cite

Improving water management in water stressed-regions requires reliable seasonal precipitation predication, which remains a grand challenge. Numerous statistical and dynamical model simulations have been developed for predicting precipitation. However, both types of models offer limited seasonal predictability. This study outlines a hybrid statistical-dynamical modeling framework for predicting seasonal precipitation. The dynamical component relies on the physically based North American Multi-Model Ensemble (NMME) model simulations (99 ensemble members). The statistical component relies on a multivariate Bayesian-based model that relates precipitation to atmosphere-ocean teleconnections (also known as an analog-year statistical model). Here, the Pacific Decadal Oscillation (PDO), Multivariate ENSO Index (MEI), and Atlantic Multi-decadal Oscillation (AMO) are used in the statistical component. The dynamical and statistical predictions are linked using the so-called Expert Advice algorithm, which offers an ensemble response (as an alternative to the ensemble mean). The latter part leads to the best precipitation prediction based on contributing statistical and dynamical ensembles. It combines the strength of physically based dynamical simulations and the capability of an analog-year model. An application of the framework in the southwestern United States, which has suffered from major droughts over the past decade, improves seasonal precipitation predictions (3-to 5-month lead time) by 5-60 percent relative to the NMME simulations. Overall, the hybrid framework performs better in predicting negative precipitation anomalies (10-60% improvement over NMME) than positive precipitation anomalies (5-25% improvement over NMME). The results indicate that the framework would likely improve our ability to predict droughts such as the 2012-2014 event in the western United States that resulted in significant socio-economic impacts.

show abstract

Competitive On‐line Statistics

Cited by 182 publications

References 51 publications

Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots

Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots

10.1162/1532443041827952

A hybrid statistical‐dynamical framework for meteorological drought prediction: Application to the southwestern United States

Contact Info

Product

Resources

About