Bipartite invariance in mouse primary visual cortex

Abstract

A defining characteristic of intelligent systems, whether natural or artificial, is the ability to generalize and infer behaviorally relevant latent causes from high-dimensional sensory input, despite significant variations in the environment. To understand how brains achieve generalization, it is crucial to identify the features to which neurons respond selectively and invariantly. However, the high-dimensional nature of visual inputs, the non-linearity of information processing in the brain, and limited experimental time make it challenging to systematically characterize neuronal tuning and invariances, especially for natural stimuli. Here, we extended "inception loops" — a paradigm that iterates between large-scale recordings, neural predictive models, andin silicoexperiments followed byin vivoverification — to systematically characterize single neuron invariances in the mouse primary visual cortex. Using the predictive model we synthesized Diverse Exciting Inputs (DEIs), a set of inputs that differ substantially from each other while each driving a target neuron strongly, and verified these DEIs' efficacyin vivo. We discovered a novel bipartite invariance: one portion of the receptive field encoded phase-invariant texture-like patterns, while the other portion encoded a fixed spatial pattern. Our analysis revealed that the division between the fixed and invariant portions of the receptive fields aligns with object boundaries defined by spatial frequency differences present in highly activating natural images. These findings suggest that bipartite invariance might play a role in segmentation by detecting texture-defined object boundaries, independent of the phase of the texture. We also replicated these bipartite DEIs in the functional connectomics MICrONs data set, which opens the way towards a circuit-level mechanistic understanding of this novel type of invariance. Our study demonstrates the power of using a data-driven deep learning approach to systematically characterize neuronal invariances. By applying this method across the visual hierarchy, cell types, and sensory modalities, we can decipher how latent variables are robustly extracted from natural scenes, leading to a deeper understanding of generalization.

A chromatic feature detector in the retina signals visual context changes

“…A remedy for this limitation is to search for diverse exciting inputs by generating stimuli that are both highly effective at eliciting neural responses and at the same time distinct from one another. Ding et al (49) used this approach to study bipartite invariance in mouse V1 (see also (50)). Related to this, Goldin et al (51) searched for locally optimal stimulus perturbations for mouse RGCs and found that the selectivity for positive or negative contrast in a subset of cells is context-dependent.…”

Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks

“…recurrence, top-down processing streams) that account for dynamical processing in the brain [ 59 ]. Moreover, our multi-task modeling approach, together with in-silico experimentation, promises to facilitate the generation of hypotheses about tuning directions via maximally exciting inputs (MEIs) [ 10 , 11 , 60 , 61 ], diverse exciting inputs (DEIs) [ 62 , 63 ], or controversial stimuli [ 49 ].…”