The geometry of representational drift in natural and artificial neural networks

Abstract

Neurons in sensory areas encode/represent stimuli. Surprisingly, recent studies have suggested that, even during persistent performance, these representations are not stable and change over the course of days and weeks. We examine stimulus representations from fluorescence recordings across hundreds of neurons in the visual cortex using in vivo two-photon calcium imaging and we corroborate previous studies finding that such representations change as experimental trials are repeated across days. This phenomenon has been termed “representational drift”. In this study we geometrically characterize the properties of representational drift in the primary visual cortex of mice in two open datasets from the Allen Institute and propose a potential mechanism behind such drift. We observe representational drift both for passively presented stimuli, as well as for stimuli which are behaviorally relevant. Across experiments, the drift differs from in-session variance and most often occurs along directions that have the most in-class variance, leading to a significant turnover in the neurons used for a given representation. Interestingly, despite this significant change due to drift, linear classifiers trained to distinguish neuronal representations show little to no degradation in performance across days. The features we observe in the neural data are similar to properties of artificial neural networks where representations are updated by continual learning in the presence of dropout, i.e. a random masking of nodes/weights, but not other types of noise. Therefore, we conclude that a potential reason for the representational drift in biological networks is driven by an underlying dropout-like noise while continuously learning and that such a mechanism may be computational advantageous for the brain in the same way it is for artificial neural networks, e.g. preventing overfitting.

Unconscious manipulation of conceptual representations with decoded neurofeedback impacts search behaviour

“…oriented gratings) (Marks & Goard, 2021). Importantly, representational drift can occur despite sustained decoding accuracy of the target content from neural responses by a linear classifier (Aitken et al, 2022), as we observed in the present study.…”

“…Representational drift has been observed in the hippocampus (Ziv et al, 2013) and also in cortical regions (Driscoll, Pettit, Minderer, Chettih, & Harvey, 2017), including sensory cortex (Aitken, Garrett, Olsen, & Mihalas, 2022; Deitch et al, 2021), when observers are exposed to similar stimuli across sessions and days. Representational drift is higher for more complex representations associated with naturalistic stimuli relative to simple items (i.e.…”

Stimulus novelty uncovers coding diversity in visual cortical circuits

“…Representational drift (Aitken et al, 2022; Deitch et al, 2021; Marks and Goard, 2021; Rule et al, 2019; Schoonover et al, 2021) could also contribute to differences in coding across days. However, several control analyses comparing cluster sizes to shuffled conditions (Extended Data Fig.…”

Few-shot pattern detection by transient boosting of somato-dendritic coupling

“…In the brain, assemblies of tightly connected neurons can compete for simple and robust representations of information carried by stimuli. As the identity of the assembly may change over time, which neuronal assembly gets recruited and wins over the rest may not matter 78,79 .…”