The AGINAO Self-Programming Engine

Abstract

The AGINAO is a project to create a human-level artificial general intelligence system (HL AGI) embodied in the Aldebaran Robotics' NAO humanoid robot.The dynamical and open-ended cognitive engine of the robot is represented by an embedded and multi-threaded control program, that is self-crafted rather than hand-crafted, and is executed on a simulated Universal Turing Machine (UTM). The actual structure of the cognitive engine emerges as a result of placing the robot in a natural preschool-like environment and running a core start-up system that executes self-programming of the cognitive layer on top of the core layer. The data from the robot's sensory devices supplies the training samples for the machine learning methods, while the commands sent to actuators enable testing hypotheses and getting a feedback. The individual self-created subroutines are supposed to reflect the patterns and concepts of the real world, while the overall program structure reflects the spatial and temporal hierarchy of the world dependencies. This paper focuses on the details of the self-programming approach, limiting the discussion of the applied cognitive architecture to a necessary minimum.

Evaluating actuators in a purely information-theory based reward model

“…The effect of suppression is currently implemented by analogy to the method of estimating the probability of exploration 3 , presented in [1]. Let's assume that for an actuator T there exist copies T 1 ,...,T n , n≥0, with the current value estimates A 1 ,...,A n .…”

“…A detailed presentation of the construction of the AGINAO self-programming engine is presented in [1]. What follows is an excerpt essential for the presentation of the paper thesis.…”

“…Let Χ i,to = C i (x 1 ,...,x n )/β at time t 0 , i.e., the time 2 the actuator T i was activated. Χ i,to is just the cost recorded at time t 0 and expressed in bits, i.e., the cost related to the current input vector 1 It is a vector of vectors of integers. 2 Again, we are using alternately the notion of time t as time-step of MDP, or as real-time.…”

“…For regular concepts, the value Q i,t of action a i at time t is updated according to the following TD-learning rule: (5) where r i,t+1 is the immediate reward at time t+1, α is the learning rate, γ is the discount factor, and V is the weighted average of values of all actions of the concept the action a i points to, computed from the equation: (6) Where p i is defined by (1). The probability of selecting action a i at time t is defined as: (7) The above definition of TD-learning rule is not applicable for the actuators for many reasons: (a) actuator-concepts do not contain a codelet that computes the immediate-reward, (b) the actuator-concepts do not really partition the state-space, but merely function as a physical-actuator proxy, (c) the computation of the weighted average value V is impossible due to the fact, that actuator-concepts are terminal-leaves and do not connect to next actions/descendents.…”

Binary Space Partitioning as Intrinsic Reward

“…It was first introduced in [4]. A more detail presentation of the self-programming engine is given in [5]. The open-ended learning, executed by the cognitive engine, is a result of a fully automatic self-programming development of a hierarchy of interconnected concepts, as shown in Fig.…”