CNN MouseNet: A biologically constrained convolutional neural network model for mouse visual cortex

Abstract

Convolutional neural networks trained on object recognition derive inspiration from the neural architecture of the visual system in primates, and have been used as models of the feedforward computation performed in the primate ventral stream. In contrast to the deep hierarchical organization of primates, the visual system of the mouse has a shallower arrangement. Since mice and primates are both capable of visually guided behavior, this raises questions about the role of architecture in neural computation. In this work, we introduce a novel framework for building a biologically constrained convolutional neural network model of the mouse visual cortex. The architecture and structural parameters of the network are derived from experimental measurements, specifically the 100-micrometer resolution interareal connectome, the estimates of numbers of neurons in each area and cortical layer, and the statistics of connections between cortical layers. This network is constructed to support detailed task-optimized models of mouse visual cortex, with neural populations that can be compared to specific corresponding populations in the mouse brain. Using a well-studied image classification task as our working example, we demonstrate the computational capability of this mouse-sized network. Given its relatively small size, MouseNet achieves roughly 2/3rds the performance level on ImageNet as VGG16. In combination with the large scale Allen Brain Observatory Visual Coding dataset, we use representational similarity analysis to quantify the extent to which MouseNet recapitulates the neural representation in mouse visual cortex. Importantly, we provide evidence that optimizing for task performance does not improve similarity to the corresponding biological system beyond a certain point. We demonstrate that the distributions of some physiological quantities are closer to the observed distributions in the mouse brain after task training. We encourage the use of the MouseNet architecture by making the code freely available.

Recurrent Connections in the Primate Ventral Visual Stream Mediate a Tradeoff Between Task Performance and Network Size During Core Object Recognition

“…Finally, we found that lowering the resolution of the inputs during model training led to improved correspondence with the neural responses across model architectures, including current deep CNNs (VGG16 and ResNet-18) used in prior comparisons to mouse visual data [26,211,47,43]. Thus, strong constraints even at the level of input transformations improve model correspondence to the mouse visual system, although there remains a small gap between these models and the inter-animal consistency ceiling.…”

“…We found that the neural response predictions of a standard deep CNN model (VGG16), used in prior comparisons to mouse visual areas [26,211,47], were quite far from the inter-animal consistency (56.27%). Retraining this model with images of resolution closer to the visual acuity of mice (64×64 pixels) improved the model's neural predictivity, reaching 67.7% of the inter-animal consistency.…”

“…Although the input resolution of 64 × 64 pixels may not be optimal for different architectures, it is the resolution that we used to train all the models. This was motivated by the observation that the upper bound on mouse visual acuity is 0.5 cycles / degree [188], corresponding to 2 pixels / cycle × 0.5 cycles / degree = 1 pixel / degree, so that a simplified correction for the appropriate visual acuity would correspond to 64 × 64 pixels, which was also used in [211] and in the MouseNet of [210]. A more thorough investigation into the appropriate image transformations, however, may be needed.…”

Neural Regression, Representational Similarity, Model Zoology & Neural Taskonomy at Scale in Rodent Visual Cortex

“…In summary, only novel combinations of architecture, task and mapping will help to explain the highly reliable neural variance we’ve yet to explain in our current survey. Already this recombination is under way: Shi et al [97] have created a custom CNN designed specifically to match (processing stage by processing stage) the anatomy of rodent visual cortex, while Nayebi et al [88] have combined the power of self-supervised learning with smaller, shallower architectures to more fully account for the ethological realities of rodent behavior and the differences in computational bandwidth that shape and constrain their visual systems. More work of this variety will be necessary to more fully model the rich diversity and fiendish complexity of biological brains at scale – even the very smallest ones.…”

“…Best practice for this interfacing remains unclear in the absence of more comprehensive empirical controls. The use of custom models (as in Nayebi et al [88] and Shi et al [97]), on the other hand, does not carry with it the same concerns as the use of pretrained models, and seems a promising path forward for probing the effects of input manipulations directly.…”