A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Irie, Kazuki; Schlag, Imanol; Csordás, Róbert; Schmidhuber, Jürgen

doi:10.48550/arxiv.2202.05780

Cited by 4 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The meta-representation refers to the representation of meta-knowledge ω. This knowledge could be anything from initial model parameters (Finn et al, 2017;Rothfuss et al, 2018;Fakoor et al, 2019;Liu et al, 2019), the inner optimization process (Andrychowicz et al, 2016;Bello et al, 2017;Metz et al, 2018;Irie et al, 2022), or the model architecture (Zoph and Le, 2016;Liu et al, 2018;Lian et al, 2019;Real et al, 2019). The meta-optimizer refers to the choice of optimization for the outer-level in the meta-training phase which updates meta-knowledge ω.…”

Section: Learning In Network With Plastic Synapses Learning How To Le...mentioning

confidence: 99%

Meta-SpikePropamine: learning to learn with synaptic plasticity in spiking neural networks

Schmidgall

Hays

2023

Front. Neurosci.

View full text Add to dashboard Cite

We propose that in order to harness our understanding of neuroscience toward machine learning, we must first have powerful tools for training brain-like models of learning. Although substantial progress has been made toward understanding the dynamics of learning in the brain, neuroscience-derived models of learning have yet to demonstrate the same performance capabilities as methods in deep learning such as gradient descent. Inspired by the successes of machine learning using gradient descent, we introduce a bi-level optimization framework that seeks to both solve online learning tasks and improve the ability to learn online using models of plasticity from neuroscience. We demonstrate that models of three-factor learning with synaptic plasticity taken from the neuroscience literature can be trained in Spiking Neural Networks (SNNs) with gradient descent via a framework of learning-to-learn to address challenging online learning problems. This framework opens a new path toward developing neuroscience inspired online learning algorithms.

show abstract

Section: Learning In Network With Plastic Synapses Learning How To Le...mentioning

confidence: 99%

Meta-SpikePropamine: learning to learn with synaptic plasticity in spiking neural networks

Schmidgall

Hays

2023

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…The meta-representation refers to the representation of meta-knowledge ω. This knowledge could be anything from initial model parameters (25)(26)(27)(28), the inner optimization process (29)(30)(31)(32), or the model architecture (33)(34)(35)(36). The meta-optimizer refers to the choice of optimization for the outer-level in the meta-training phase which updates meta-knowledge ω.…”

Section: Learning In Network With Plastic Synapsesmentioning

confidence: 99%

Learning to learn online with neuromodulated synaptic plasticity in spiking neural networks

Schmidgall

Hays

2022

Preprint

View full text Add to dashboard Cite

We propose that in order to harness our understanding of neuroscience toward machine learning, we must first have powerful tools for training brain-like models of learning. Although substantial progress has been made toward understanding the dynamics of learning in the brain, neuroscience-derived models of learning have yet to demonstrate the same performance capabilities as methods in deep learning such as gradient descent. Inspired by the successes of machine learning using gradient descent, we demonstrate that models of neuromodulated synaptic plasticity from neuroscience can be trained in Spiking Neural Networks (SNNs) with a framework of learning to learn through gradient descent to address challenging online learning problems. This framework opens a new path toward developing neuroscience inspired online learning algorithms.

show abstract

“…We visualize their attention maps in Section VI-B. The observed generalization gap makes environments with large differences in object distributions (requiring zero-shot adaptation) fruitful for developing and evaluating novel fastadaptation [44]- [48] and meta-learning [49] agents, such as the ones based on Fast Weight Programmers [50]- [54].…”

Section: Ood Generalization Experimentsmentioning

confidence: 99%

Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter

Stanić¹,

Tang²,

Ha³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reinforcement learning agents must generalize beyond their training experience. Prior work has focused mostly on identical training and evaluation environments. Starting from the recently introduced Crafter benchmark, a 2D open world survival game, we introduce a new set of environments suitable for evaluating some agent's ability to generalize on previously unseen (numbers of) objects and to adapt quickly (meta-learning). In Crafter, the agents are evaluated by the number of unlocked achievements (such as collecting resources) when trained for 1M steps. We show that current agents struggle to generalize, and introduce novel object-centric agents that improve over strong baselines. We also provide critical insights of general interest for future work on Crafter through several experiments. We show that careful hyper-parameter tuning improves the PPO baseline agent by a large margin and that even feedforward agents can unlock almost all achievements by relying on the inventory display. We achieve new state-of-the-art performance on the original Crafter environment. Additionally, when trained beyond 1M steps, our tuned agents can unlock almost all achievements. We show that the recurrent PPO agents improve over feedforward ones, even with the inventory information removed. We introduce CrafterOOD, a set of 15 new environments that evaluate OOD generalization. On CrafterOOD, we show that the current agents fail to generalize, whereas our novel objectcentric agents achieve state-of-the-art OOD generalization while also being interpretable. Our code is public. 1

show abstract

A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Cited by 4 publications

References 21 publications

Meta-SpikePropamine: learning to learn with synaptic plasticity in spiking neural networks

Meta-SpikePropamine: learning to learn with synaptic plasticity in spiking neural networks

Learning to learn online with neuromodulated synaptic plasticity in spiking neural networks

Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter

Contact Info

Product

Resources

About