Reflective Variants of Solomonoff Induction and AIXI

Fallenstein, Benja; Soares, Nate; Taylor, Jessica

doi:10.1007/978-3-319-21365-1_7

Cited by 3 publications

(1 citation statement)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The central idea to our construction is based on reflective oracles [FST15,FTC15b]. Reflective oracles are probabilistic oracles similar to halting oracles that answer whether the probability that a given probabilistic Turing machine T outputs 1 is higher than a given rational number p. The oracles are reflective in the sense that the machine T may itself query the oracle, so the oracle has to answer queries about itself.…”

Section: Reinforcement Learningmentioning

confidence: 99%

A Formal Solution to the Grain of Truth Problem

Leike¹,

Taylor²,

Fallenstein³

2016

Preprint

Self Cite

View full text Add to dashboard Cite

A Bayesian agent acting in a multi-agent environment learns to predict the other agents' policies if its prior assigns positive probability to them (in other words, its prior contains a grain of truth). Finding a reasonably large class of policies that contains the Bayes-optimal policies with respect to this class is known as the grain of truth problem. Only small classes are known to have a grain of truth and the literature contains several related impossibility results. In this paper we present a formal and general solution to the full grain of truth problem: we construct a class of policies that contains all computable policies as well as Bayes-optimal policies for every lower semicomputable prior over the class. When the environment is unknown, Bayes-optimal agents may fail to act optimally even asymptotically. However, agents based on Thompson sampling converge to play ε-Nash equilibria in arbitrary unknown computable multi-agent environments. While these results are purely theoretical, we show that they can be computationally approximated arbitrarily closely.

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

A Formal Solution to the Grain of Truth Problem

Leike¹,

Taylor²,

Fallenstein³

2016

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Reflective Oracles: A Foundation for Game Theory in Artificial Intelligence

Fallenstein

Taylor

Christiano

2015

Logic, Rationality, and Interaction

Self Cite

View full text Add to dashboard Cite

Classical game theory treats players as special-a description of a game contains a full, explicit enumeration of all players-even though in the real world, "players" are no more fundamentally special than rocks or clouds. It isn't trivial to find a decision-theoretic foundation for game theory in which an agent's coplayers are a non-distinguished part of the agent's environment. Attempts to model both players and the environment as Turing machines, for example, fail for standard diagonalization reasons.

show abstract

Nonparametric General Reinforcement Learning

Leike¹

2016

View full text Add to dashboard Cite

Reflective Variants of Solomonoff Induction and AIXI

Cited by 3 publications

References 5 publications

A Formal Solution to the Grain of Truth Problem

A Formal Solution to the Grain of Truth Problem

Reflective Oracles: A Foundation for Game Theory in Artificial Intelligence

Nonparametric General Reinforcement Learning

Contact Info

Product

Resources

About