Abstract. Recent years have seen a resurgence of interest in evolving plastic neural networks for online learning. These approaches have an intrinsic appeal -since, to date, the only working example of general intelligence is the human brain, which has developed through evolution, and exhibits a great capacity to adapt to unfamiliar environments. In this paper we review prior work in this area -including problem domains and tasks, fitness functions, synaptic plasticity models and neural network encoding schemes. We conclude with a discussion of current findings and promising future directions, including incorporation of functional properties observed in biological neural networks which appear to play a role in learning processes, and addressing the "general" in general intelligence by the introduction of previously unseen tasks during the evolution process.

Algorithms for evolving agents that learn during their lifetime have typically been evaluated on only a handful of environments. Designing such environments is labour intensive, potentially biased, and provides only a small sample size that may prevent accurate general conclusions from being drawn. In this paper we introduce a method for automatically generating MDP environments which allows the difficulty to be scaled in several ways. We present a case study in which environments are generated that vary along three key dimensions of difficulty: the number of environment configurations, the number of available actions, and the length of each trial. The study reveals interesting differences between three neural network models -Fixed-Weight, Plastic-Weight, and Modulated Plasticity -that would not have been obvious without sweeping across these different dimensions. Our paper thus introduces a new way of conducting reinforcement learning science: instead of manually designing a few environments, researchers will be able to automatically generate a range of environments across key dimensions of variation. This will allow scientists to more rigorously assess the general learning capabilities of an algorithm, and may ultimately improve the rate at which we discover how to create AI with general purpose learning.