Fast Adaptation to Rule Switching using Neuronal Surprise

Abstract

In humans and animals, surprise is a physiological reaction to an unexpected event, but how surprise can be linked to plausible models of neuronal activity is an open problem. We propose a self-supervised spiking neural network model where a surprise signal is extracted from an increase in neural activity after an imbalance of excitation and inhibition. The surprise signal modulates synaptic plasticity via a three-factor learning rule which increases plasticity at moments of surprise. The surprise signal remains small when transitions between sensory events follow a previously learned rule but increases immediately after rule switching. In our model, previously learned rules are protected against overwriting when learning a new rule. Our surprise-modulated spiking network model makes a step towards solving the stability-plasticity dilemma in neuroscience and the problem of continual learning in machine learning.

P3-like signatures of temporal predictions: a computational EEG study

“…Although such a model was designed to capture all the peculiarities of the task (see O'Reilly et al, 2013), it represents "just a description of optimal behavior: it does not prescribe how Bayes optimal perception, sensorimotor integration or decision-making under uncertainty emerges" (Friston, 2012). A full understanding of how the brain updates temporal expectations needs more sophisticated models that consider the constraints deriving from the anatomy and the physiology of the brain, such as its functional hierarchical architecture or neuronal dynamics (e.g., Barry & Gerstner, 2022;Iigaya, 2016). In this respect, the fact that the use of an alternative and more biologically plausible model (i.e., the HGF; Mathys et al, 2011) led to very similar results corroborates our conclusions.…”

Exploratory exploitation and exploitative exploration: The phenomenology of play and the computational dynamics of search

“…Surprise signals, on the contrary, arise in the face of unexpected stimuli, both familiar and unfamiliar ones . In line with that, different neuromodulatory signals are thought to communicate expected versus unexpected novelty or uncertainty Yu & Dayan, 2005); and computational models suggest different network mechanisms for the detection of novelty and surprise (Barry & Gerstner, 2024;Schulz et al, 2021). Despite the partial overlap in the processing of novelty and surprise , we can thus not simply speak of "novelty" detection as a homogeneous process as assumed in the NSM.…”

Computational models of intrinsic motivation for curiosity and creativity

“…Surprise signals, on the contrary, arise in the face of unexpected stimuli, both familiar and unfamiliar ones (Zhang et al, 2022). In line with that, different neuromodulatory signals are thought to communicate expected versus unexpected novelty or uncertainty (Schomaker & Meeter, 2015; Yu & Dayan, 2005); and computational models suggest different network mechanisms for the detection of novelty and surprise (Barry & Gerstner, 2024; Schulz et al, 2021). Despite the partial overlap in the processing of novelty and surprise (Zhang et al, 2022), we can thus not simply speak of “novelty” detection as a homogeneous process as assumed in the NSM.…”