The Geometry of Representational Drift in Natural and Artificial Neural Networks

Abstract

Neurons in sensory areas encode/represent stimuli. Surprisingly, recent studies have suggest that, even during persistent performance, these representations are not stable and change over the course of days and weeks. We examine stimulus representations from fluorescence recordings across hundreds of neurons in the visual cortex using in vivo two-photon calcium imaging and we corroborate previous studies finding that such representations change as experimental trials are repeated across days. This phenomenon has been termed "representational drift". In this study we geometrically characterize the properties of representational drift in the primary visual cortex of mice in two open datasets from the Allen Institute and propose a potential mechanism behind such drift. We observe representational drift both for passively presented stimuli, as well as for stimuli which are behaviorally relevant. Across experiments, the drift most often occurs along directions that have the most variance, leading to a significant turnover in the neurons used for a given representation. Interestingly, despite this significant change due to drift, linear classifiers trained to distinguish neuronal representations show little to no degradation in performance across days. The features we observe in the neural data are similar to properties of artificial neural networks where representations are updated by continual learning in the presence of dropout, i.e. a random masking of nodes/weights, but not other types of noise. Therefore, we conclude that a potential reason for the representational drift in biological networks is driven by an underlying dropout-like noise while continuously learning and that such a mechanism may be computational advantageous for the brain in the same way it is for artificial neural networks, e.g. preventing overfitting.

Differential encoding of temporal context and expectation under representational drift across hierarchically connected areas

“…Third, we found a strong drift of population activity on the timescale of minutes within the recording session, whose features differed along the hierarchy. In V1 and PM, we found that drift, defined by a significant encoding of elapsed time within the session, also preserved the representational geometry of the evoked responses, consistent with previous experimental and modeling studies [Deitch et al, 2021, Aitken et al, 2021, Qin et al, 2023. On the other hand, RSP showed significant encoding of elapsed time within a session, but the representational geometry of evoked responses to sensory stimuli did not generalize across epochs, suggesting that relevant dimensions for RSP activity may not be stimulus specific, but potentially more related to overall environmental context.…”

Coordinated changes in a cortical circuit sculpt effects of novelty on neural dynamics

“…Our findings thus raise additional questions about the complex neural dynamics in V1, offering a basis for comparison with other modeling studies. For example, our approach contrasts with a recently published study by Aitken et al (2023), which also employs the same experimental paradigm developed by Garrett et al (2020Garrett et al ( , 2023. Aitken et al focus on elucidating a biologically plausible mechanism of synaptic plasticity, capable of reproducing novelty effects across different timescales, while also simulating individual neurons to describe their responses and diversity; however, they do not extensively seek the specific circuitry changes or their parametric details.…”

“…However, we did not include those in our modeling framework because it will require a model to have multiple channels of image inputs and a mechanism that changes the response to the images within the same condition, thus complicating the model. Besides, responses to such contextual novelty have been already described using synaptic plasticity (Aitken et al, 2023;Schulz et al, 2021). Therefore, we selected data segments around omission as the target data for our model, as these observations underscore the complex interplay between the circuit elements in V1, particularly in the interaction between absolute and omission novelty.…”

Ongoing neuronal population activity dynamics in the neocortex - representational drift in experiment and model

“…A similar mechanism is potentially helpful in the brain, too. And drift leads to similar results as dropout (Aitken et al, 2021). Sup.…”

“…This is mostly achieved by homeostatic plasticity. Random drift, however, might be useful, too, both from a psychological point of view to overcome trauma (Richards and Frankland, 2017) and from a theoretical point of view to prevent overfitting (Aitken et al, 2021). The need for stability and flexibility depends on the situation.…”