Presented topic is from area of development of artificial creatures and proposes new architecture of autonomous agent. The work builds on a research of the latest approaches to Artificial Life, realized by the Department of Cybernetics, CTU in Prague in the last twenty years. This architecture design combines knowledge from Artificial Intelligence (AI), Ethology, Artificial Life (ALife) and Intelligent Robotics. From the field of classical AI, the fusion of reinforcement learning, planning and artificial neural network into one more complex control system was used here. The main principle of its function is inspired by the field of Ethology, this means that life of given agent tries to be similar to life of an animal in the Nature, where animal learns relatively autonomously from simpler principles towards the more complex ones. The architecture supports on-line learning of all knowledge from the scratch, while the core principle is in hierarchical Reinforcement Learning (RL), this action hierarchy is created autonomously based solely on agents interaction with an environment. The main key idea behind this approach is in original implementation of a domain independent hierarchical planner. Our planner is able to operate with behaviors learned by the RL. It means that an autonomously gained hierarchy of actions can be used not only by action selection mechanisms based on the reinforcement learning, but also by a planning system. This gives the agent ability to utilize high-level deliberative problem solving based solely on his experiences. In order to deal with higher-level control rather than a sensory system, the life of agent was simulated in a virtual environment.

No abstract

No abstract

Research in Artificial Intelligence (AI) has focused mostly on two extremes: either on small improvements in narrow AI domains, or on universal theoretical frameworks which are often uncomputable, or lack practical implementations. In this paper we attempt to follow a big picture view while also providing a particular theory and its implementation to present a novel, purposely simple, and interpretable hierarchical architecture. This architecture incorporates the unsupervised learning of a model of the environment, learning the influence of one's own actions, model-based reinforcement learning, hierarchical planning, and symbolic/subsymbolic integration in general. The learned model is stored in the form of hierarchical representations which are increasingly more abstract, but can retain details when needed. We demonstrate the universality of the architecture by testing it on a series of diverse environments ranging from audio/visual compression to discrete and continuous action spaces, to learning disentangled representations. MotivationDespite the fact that strong AI capable of handling a diverse set of human-level tasks was envisioned decades ago, and there has been significant progress in developing AI for narrow tasks, we are still far away from having a single system which would be able to learn with efficiency and generality comparable to human beings or animals. While practical research has focused mostly on small improvements in narrow AI domains, research in the area of Artificial General Intelligence (AGI) has tended to focus on frameworks of truly general theories, like AIXI [1], Causal Entropic Forces [2], or PowerPlay [3]. These are usually uncomputable, incompatible with theories of biological intelligence, and/or lack practical implementations. Another class of algorithm that can be mentioned encompasses systems that are usually somewhere on the edge of cognitive architectures and adaptive general problem-solving systems. Examples of such systems are: the Non-Axiomatic Reasoning System [4], Growing Recursive Self-Improvers [5], recursive data compression architecture [6], OpenCog [7], Never-Ending Language Learning [8], Ikon Flux [9], MicroPsi [10], Lida [11] and many others [12]. These systems usually have a fixed structure with adaptive parts and are in some cases