Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations

Abstract

Temporal models based on recurrent neural networks have proven to be quite powerful in a wide variety of applications, including language modeling and speech processing. However, training these models often relies on back-propagation through time, which entails unfolding the network over many time steps, making the process of conducting credit assignment considerably more challenging. Furthermore, the nature of backpropagation itself does not permit the use of non-differentiable activation functions and is inherently sequential, making parallelization of the underlying training process difficult.Here, we propose the Parallel Temporal Neural Coding Network (P-TNCN), a biologically inspired model trained by the learning algorithm we call Local Representation Alignment. It aims to resolve the difficulties and problems that plague recurrent networks trained by back-propagation through time. The architecture requires neither unrolling in time nor the derivatives of its internal activation functions. We compare our model and learning procedure to other back-propagation through time alternatives (which also tend to be computationally expensive), including real-time recurrent learning, echo state networks, and unbiased online recurrent optimization. We show that it outperforms these on sequence modeling benchmarks such as Bouncing MNIST, a new benchmark we denote as Bouncing NotMNIST, and Penn Treebank. Notably, our approach can in some instances outperform full back-propagation through time as well as variants such as sparse attentive back-tracking.Significantly, the hidden unit correction phase of P-TNCN allows it to adapt to new datasets even if its synaptic weights are held fixed (zero-shot adaptation) and facilitates retention of prior generative knowledge when faced with a task sequence. We present results that show the P-TNCN's ability to conduct zero-shot adaptation and online continual sequence modeling.

Backprop-Free Reinforcement Learning with Active Neural Generative Coding

“…Neural generative coding (NGC) is a recently developed framework [35] that generalizes classical ideas in predictive processing [40,39] to the construction of scalable neural models that model and predict both static and temporal patterns [33,35]. An NGC model is composed of L layers of stateful neurons, where the activity values of each layer = {0, 1,…”

“…The online objective that an NGC model attempts to minimize is known as total discrepancy [44], from which the error neuron, state update expressions, and local synaptic adjustments may be derived [41,35]. The total discrepancy objective, which could also be interpreted as a form of free energy [45] specialized for the case of stateful neural models that utilize arbitrary forward and error synaptic wiring pathways [41], can be expressed in many forms including the linear combination of local density functions [35] or the summation of local distance functions [33]. For this study, the form of total discrepancy we used to derive the expressions above is the linear combination of distance functions:…”

“…While many backprop-alternative (backprop-free) algorithms have recently been proposed [25,26,27,28,29,30,31,23], few have been investigated outside the context of supervised learning, with some notable exceptions in sequence [32,33,34] and generative modeling [35]. In the realm of RL, aside from neuro-evolutionary approaches [36,37], the dearth of work is more prescient and it is our intent to close this gap by providing a backprop-free approach to inference and learning, which we call active neural generative coding (ANGC), to drive goal-oriented agents.…”

Continual Competitive Memory

“…NotMNIST 5 is a more difficult variation of MNIST created by replacing the digits with characters of varying fonts/glyphs (letters A-J). The preprocessing and training, validation, and testing splits were created to match the setup of MNIST with the exception that there is more data in NotMNIST, i.e., 100, 000 data points are in the training split we used for our experiments (adapted from the smaller variant in [28]).…”

NIC: An innovative supervised machine learning computational framework having network‐based interactional connections

“…For example, the ability of neural networks to predict wind speed 6 or neuro‐FS to predict the magnitude of the next large earthquakes could be considered 7 . There are so many types of SC models that introduced, improved, or evaluated in the literature 8–12 to resolve the complex problems in different fields such as nonlinear delayed systems, 13 transcoding simultaneously acquired MRI data, 14 power system restoration, 15 eye surgery, 16 broiler output energies, 17 intelligent human action recognition, 18 behavior and environmental impacts of waste glass geopolymers, 19 and also for concrete elements 20–23 …”