Binary Space Partitioning as Intrinsic Reward

Skaba, Wojciech

doi:10.1007/978-3-642-35506-6_29

Cited by 4 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The following discussion of reinforcement-learning and binary space partitioning is an abbreviated version of a more detailed presentation by Skaba (2012).…”

Section: Evaluating the Conceptsmentioning

confidence: 99%

The AGINAO Self-Programming Engine

Skaba¹

2013

Journal of Artificial General Intelligence

Self Cite

View full text Add to dashboard Cite

The AGINAO is a project to create a human-level artificial general intelligence system (HL AGI) embodied in the Aldebaran Robotics' NAO humanoid robot.The dynamical and open-ended cognitive engine of the robot is represented by an embedded and multi-threaded control program, that is self-crafted rather than hand-crafted, and is executed on a simulated Universal Turing Machine (UTM). The actual structure of the cognitive engine emerges as a result of placing the robot in a natural preschool-like environment and running a core start-up system that executes self-programming of the cognitive layer on top of the core layer. The data from the robot's sensory devices supplies the training samples for the machine learning methods, while the commands sent to actuators enable testing hypotheses and getting a feedback. The individual self-created subroutines are supposed to reflect the patterns and concepts of the real world, while the overall program structure reflects the spatial and temporal hierarchy of the world dependencies. This paper focuses on the details of the self-programming approach, limiting the discussion of the applied cognitive architecture to a necessary minimum.

show abstract

“…The following discussion of reinforcement-learning and binary space partitioning is an abbreviated version of a more detailed presentation by Skaba (2012).…”

Section: Evaluating the Conceptsmentioning

confidence: 99%

The AGINAO Self-Programming Engine

Skaba¹

2013

Journal of Artificial General Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…The application of binary space-partitioning for the estimation of the immediate-reward was first introduced in [6]. Fig.…”

Section: B Binary Space-partitioning and Computation Of The Rewardmentioning

confidence: 99%

“…The mean reward, rather than the reward provided by a positive example, is interpreted as immediate-reward for TDlearning, for we want to estimate the value of a concept. Self-information is also a special case of Kullback-Leibler distance from a Kronecker delta representing the matching pattern to the probability distribution [6], [8].…”

Section: B Binary Space-partitioning and Computation Of The Rewardmentioning

confidence: 99%

“…For regular concepts, the value Q i,t of action a i at time t is updated according to the following TD-learning rule: (5) where r i,t+1 is the immediate reward at time t+1, α is the learning rate, γ is the discount factor, and V is the weighted average of values of all actions of the concept the action a i points to, computed from the equation: (6) Where p i is defined by (1). The probability of selecting action a i at time t is defined as: (7) The above definition of TD-learning rule is not applicable for the actuators for many reasons: (a) actuator-concepts do not contain a codelet that computes the immediate-reward, (b) the actuator-concepts do not really partition the state-space, but merely function as a physical-actuator proxy, (c) the computation of the weighted average value V is impossible due to the fact, that actuator-concepts are terminal-leaves and do not connect to next actions/descendents.…”

Section: B Td-learning Rule For Actuatormentioning

confidence: 99%

“…According to a convention assumed in[1] and[6], we would use the name exploration for adding a new action, and exploitation for selecting any available action, not only the most greedy one.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Evaluating actuators in a purely information-theory based reward model

Skaba¹

2013

2013 IEEE Symposium on Computational Intelligence for Human-Like Intelligence (CIHLI)

Self Cite

View full text Add to dashboard Cite

AGINAO builds its cognitive engine by applying selfprogramming techniques to create a hierarchy of interconnected codelets -the tiny pieces of code executed on a virtual machine. These basic processing units are evaluated for their applicability and fitness with a notion of reward calculated from selfinformation gain of binary partitioning of the codelet's input state-space. This approach, however, is useless for the evaluation of actuators. Instead, a model is proposed in which actuators are evaluated by measuring the impact that an activation of an effector, and consequently the feedback from the robot sensors, has on average reward received by the processing units.

show abstract

Universal Induction with Varying Sets of Combinators

Potapov¹,

Родионов²

2013

Artificial General Intelligence

View full text Add to dashboard Cite

Universal induction is a crucial issue in AGI. Its practical applicability can be achieved by the choice of the reference machine or representation of algorithms agreed with the environment. This machine should be updatable for solving subsequent tasks more efficiently. We study this problem on an example of combinatory logic as the very simple Turing-complete reference machine, which enables modifying program representations by introducing different sets of primitive combinators. Genetic programming system is used to search for combinator expressions, which are easily decomposed into sub-expressions being recombined in crossover. Our experiments show that low-complexity induction or prediction tasks can be solved by the developed system (much more efficiently than using brute force); useful combinators can be revealed and included into the representation simplifying more difficult tasks. However, optimal sets of combinators depend on the specific task, so the reference machine should be adaptively chosen in coordination with the search engine.

show abstract

Binary Space Partitioning as Intrinsic Reward

Cited by 4 publications

References 5 publications

The AGINAO Self-Programming Engine

The AGINAO Self-Programming Engine

Evaluating actuators in a purely information-theory based reward model

Universal Induction with Varying Sets of Combinators

Contact Info

Product

Resources

About