Multifunctionality in a Reservoir Computer

Abstract

Multifunctionality is a well observed phenomenological feature of biological neural networks and considered to be of fundamental importance to the survival of certain species over time. These multifunctional neural networks are capable of performing more than one task without changing any network connections. In this paper we investigate how this neurological idiosyncrasy can be achieved in an artificial setting with a modern machine learning paradigm known as 'Reservoir Computing'. A training technique is designed to enable a Reservoir Computer to perform tasks of a multifunctional nature. We explore the critical effects that changes in certain parameters can have on the Reservoir Computers' ability to express multifunctionality. We also expose the existence of several 'untrained attractors'; attractors which dwell within the prediction state space of the Reservoir Computer that were not part of the training. We conduct a bifurcation analysis of these untrained attractors and discuss the implications of our results.Advancements in Machine Learning often arise from a 'two-way street' between neuroscientific observation and mathematical representation. In this paper we stroll through such a street with inspiration from 'Multifunctional Neural Networks'. These are networks of neurons whose activity patterns can change on the demand of performing a given duty but synapses remain fixed. We conceptualise multifunctionality in the context of Dynamical Systems and Machine Learning by using a Reservoir Computer as a means to realise this neurological feat in an artificial setting. More specifically, we train a Reservoir Computer to imitate the dynamics of numerous chaotic attractors from different sources, based on a given initial condition. To do this we design a training technique which 'blends' and weights data from these chaotic attractors. We explore how different weightings and changes in the memory of this artificial neural network effect the desired learning outcomes. In doing so we uncover some 'behindthe-scenes' bifurcations of several other attractors found to be lurking within the prediction state space that interfere with the networks capacity to express multifunctionality. Above all, this paper identifies some new application areas suitable to a Reservoir Computer and broadens the current understanding of the dynamical capabilities inherent to this learning system.