Replay in Deep Learning: Current Approaches and Missing Biological Elements

Hayes, Tyler L.; Bazhenov, Maxim; Siegelmann, Hava T.; Sejnowski, Terrence J.; Kanan, Christopher

doi:10.48550/arxiv.2104.04132

Cited by 7 publications

(9 citation statements)

References 240 publications

(323 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This process of buffering data, or experiences, which are later replayed to cortical networks is precisely the role ascribed to the hippocampus in the complementary learning systems theory [51]. Our work therefore adds meta-learning to the repertoire of functions for which replay might be critical [52].…”

Section: Discussionmentioning

confidence: 84%

A contrastive rule for meta-learning

Zucchet¹,

Schug²,

Oswald³

et al. 2021

Preprint

View full text Add to dashboard Cite

Meta-learning algorithms leverage regularities that are present on a set of tasks to speed up and improve the performance of a subsidiary learning process. Recent work on deep neural networks has shown that prior gradient-based learning of meta-parameters can greatly improve the efficiency of subsequent learning. Here, we present a biologically plausible meta-learning algorithm based on equilibrium propagation. Instead of explicitly differentiating the learning process, our contrastive meta-learning rule estimates meta-parameter gradients by executing the subsidiary process more than once. This avoids reversing the learning dynamics in time and computing second-order derivatives. In spite of this, and unlike previous first-order methods, our rule recovers an arbitrarily accurate meta-parameter update given enough compute. We establish theoretical bounds on its performance and present experiments on a set of standard benchmarks and neural network architectures.

show abstract

Section: Discussionmentioning

confidence: 84%

A contrastive rule for meta-learning

Zucchet¹,

Schug²,

Oswald³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, a two stages consolidation can be envisaged where a quick (real-time) update takes place at the output layer and a slower (but more precise knowledge organization) is performed in background affecting deeper layers. This is also in line with biological learning where the hippocampus can make recent knowledge immediately accessible while consolidation in the cortex is carried out throughout sleep-awake cycles [6]. The application is equipped with a MobileNetV1-1.0 in 32-bit floating point format working with 128x128 input images [7] and leverages a fixed-size replay buffer of just 500 patterns for which only latent representations are kept.…”

Section: Core: An Android Application Based On Caffementioning

confidence: 75%

Continual Learning at the Edge: Real-Time Training on Smartphone Devices

Pellegrini¹,

Lomonaco²,

Graffieti³

et al. 2021

Preprint

View full text Add to dashboard Cite

On-device training for personalized learning is a challenging research problem. Being able to quickly adapt deep prediction models at the edge is necessary to better suit personal user needs. However, adaptation on the edge poses some questions on both the efficiency and sustainability of the learning process and on the ability to work under shifting data distributions. Indeed, naively fine-tuning a prediction model only on the newly available data results in catastrophic forgetting, a sudden erasure of previously acquired knowledge. In this paper, we detail the implementation and deployment of a hybrid continual learning strategy (AR1*) on a native Android application for real-time on-device personalization without forgetting. Our benchmark, based on an extension of the CORe50 dataset, shows the efficiency and effectiveness of our solution.

show abstract

“…Several approaches for this kind of "Continual Learning" problem [1] have been developed in the A.I. space, including: dynamic architectures that can grow network capacity, weight regularization-based approaches that mitigate catastrophic forgetting by constraining the update of previous weights [2], [3], and interleaved replay of training examples from previous tasks [4]. In contrast to rote replay (e.g., using a memory buffer to store oneto-one copies of experience), generative replay [4] does not store exact copies of specific examples from previous tasks, but instead trains a network to retain higher-level/compressed representations, from which it can create de novo synthetic training samples.…”

Section: Introductionmentioning

confidence: 99%

Continual learning benefits from multiple sleep stages: NREM, REM, and Synaptic Downscaling

Robinson

New

Nichols

et al. 2022

2022 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Learning new tasks and skills in succession without overwriting or interfering with prior learning (i.e., "catastrophic forgetting") is a computational challenge for both artificial and biological neural networks, yet artificial systems struggle to achieve even rudimentary parity with the performance and functionality apparent in biology. One of the processes found in biology that can be adapted for use in artificial systems is sleep, in which the brain deploys numerous neural operations relevant to continual learning and ripe for artificial adaptation. Here, we investigate how modeling three distinct components of mammalian sleep together affects continual learning in artificial neural networks: (1) a veridical memory replay process observed during non-rapid eye movement (NREM) sleep; (2) a generative memory replay process linked to REM sleep; and (3) a synaptic downscaling process which has been proposed to tune signal-tonoise ratios and support neural upkeep. To create this tripartite artificial sleep, we modeled NREM veridical replay by training the network using intermediate representations of samples from the current task. We modeled REM by utilizing a generator network to create intermediate representations of samples from previous tasks for training. Synaptic downscaling, a novel contribution, is modeled utilizing a size-dependent downscaling of network weights. We find benefits from the inclusion of all three sleep components when evaluating performance on a continual learning CIFAR-100 image classification benchmark. Maximum accuracy improved during training and catastrophic forgetting was reduced during later tasks. While some catastrophic forgetting persisted over the course of network training, higher levels of synaptic downscaling lead to better retention of early tasks and further facilitated the recovery of early task accuracy during subsequent training. One key takeaway is that there is a trade-off at hand when considering the level of synaptic downscaling to use -more aggressive downscaling better protects early tasks, but less downscaling enhances the ability to learn new tasks. Intermediate levels can strike a balance with the highest overall accuracies during training. Overall, our results both provide insight into how to adapt sleep components to enhance artificial continual learning systems and highlight areas for future neuroscientific sleep research to further such systems.

show abstract

Replay in Deep Learning: Current Approaches and Missing Biological Elements

Cited by 7 publications

References 240 publications

A contrastive rule for meta-learning

A contrastive rule for meta-learning

Continual Learning at the Edge: Real-Time Training on Smartphone Devices

Continual learning benefits from multiple sleep stages: NREM, REM, and Synaptic Downscaling

Contact Info

Product

Resources

About