AGI and the Knight-Darwin Law: Why Idealized AGI Reproduction Requires Collaboration

Lecture Notes in Computer Science

2021

Self Cite

We propose that, for the purpose of studying theoretical properties of the knowledge of an agent with Artificial General Intelligence (that is, the knowledge of an AGI), a pragmatic way to define such an agent's knowledge (restricted to the language of Epistemic Arithmetic, or EA) is as follows. We declare an AGI to know an EA-statement φ if and only if that AGI would include φ in the resulting enumeration if that AGI were commanded: "Enumerate all the EA-sentences which you know." This definition is non-circular because an AGI, being capable of practical English communication, is capable of understanding the everyday English word "know" independently of how any philosopher formally defines knowledge; we elaborate further on the non-circularity of this circular-looking definition. This elegantly solves the problem that different AGIs may have different internal knowledge definitions and yet we want to study knowledge of AGIs in general, without having to study different AGIs separately just because they have separate internal knowledge definitions. Finally, we suggest how this definition of AGI knowledge can be used as a bridge which could allow the AGI research community to import certain abstract results about mechanical knowing agents from mathematical logic.

Section: An Elegant Definition Of Mathematical Knowledgementioning

confidence: 99%

Section: Remarkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Short-Circuiting the Definition of Mathematical Knowledge for an Artificial General Intelligence

Lecture Notes in Computer Science

2021

Self Cite

“…ASPI measures like that of Definition 31, and 2. intelligence measures based on which ordinals the AGI knows (Alexander, 2019b). Indeed, Alexander (2020a) has argued that the task of notating large ordinals is one which spans the entire range of intelligence. This is reminiscent of Chaitin's proposal to use ordinal notation as a goal intended to facilitate evolution-"and the larger the ordinal, the fitter the organism" (Chaitin, 2011)-and Good's observation (Good, 1969) that iterated Lucas-Penrose contests boil down to contests to name the larger ordinal.…”

Section: A Remark About Aspi Measures and Agi Intelligencementioning

confidence: 99%

Measuring Intelligence and Growth Rate: Variations on Hibbard’s Intelligence Measure

Journal of Artificial General Intelligence

Hibbard

2021

In 2011, Hibbard suggested an intelligence measure for agents who compete in an adversarial sequence prediction game. We argue that Hibbard’s idea should actually be considered as two separate ideas: first, that the intelligence of such agents can be measured based on the growth rates of the runtimes of the competitors that they defeat; and second, one specific (somewhat arbitrary) method for measuring said growth rates. Whereas Hibbard’s intelligence measure is based on the latter growth-rate-measuring method, we survey other methods for measuring function growth rates, and exhibit the resulting Hibbard-like intelligence measures and taxonomies. Of particular interest, we obtain intelligence taxonomies based on Big-O and Big-Theta notation systems, which taxonomies are novel in that they challenge conventional notions of what an intelligence measure should look like. We discuss how intelligence measurement of sequence predictors can indirectly serve as intelligence measurement for agents with Artificial General Intelligence (AGIs).

“…Then it seems RL cannot lead to AGI, because "reward is not enough" for at least one type of human intelligence, namely RL-solving intelligence. 4. RL cannot learn itself, and humans are not capable of designing sufficiently good RL agents.…”

Section: Introductionmentioning

confidence: 99%

Can Reinforcement Learning Learn Itself? A Reply to ‘Reward is Enough’

Lecture Notes in Computer Science

2022

Self Cite

In their paper 'Reward is enough', Silver et al conjecture that the creation of sufficiently good reinforcement learning (RL) agents is a path to artificial general intelligence (AGI). We consider one aspect of intelligence Silver et al did not consider in their paper, namely, that aspect of intelligence involved in designing RL agents. If that is within human reach, then it should also be within AGI's reach. This raises the question: is there an RL environment which incentivises RL agents to design RL agents?