Unsupervised neural network models of the ventral visual stream

Abstract

Deep neural networks currently provide the best quantitative models of the response patterns of neurons throughout the primate ventral visual stream. However, such networks have remained implausible as a model of the development of the ventral stream, in part because they are trained with supervised methods requiring many more labels than are accessible to infants during development. Here, we report that recent rapid progress in unsupervised learning has largely closed this gap. We find that neural network models learned with deep unsupervised contrastive embedding methods achieve neural prediction accuracy in multiple ventral visual cortical areas that equals or exceeds that of models derived using today’s best supervised methods and that the mapping of these neural network models’ hidden layers is neuroanatomically consistent across the ventral stream. Strikingly, we find that these methods produce brain-like representations even when trained solely with real human child developmental data collected from head-mounted cameras, despite the fact that these datasets are noisy and limited. We also find that semisupervised deep contrastive embeddings can leverage small numbers of labeled examples to produce representations with substantially improved error-pattern consistency to human behavior. Taken together, these results illustrate a use of unsupervised learning to provide a quantitative model of a multiarea cortical brain system and present a strong candidate for a biologically plausible computational theory of primate sensory learning.

From photos to sketches - how humans and deep neural networks process objects across different levels of visual abstraction

“…While the generalization capabilities in human recognition are still out of reach for networks with a feedforward architecture and supervised learning regime (Geirhos et al, 2019;Geirhos, Janssen, et al, 2018;Geirhos, Temme, et al, 2018), developing models that more closely match the human brain in terms of architecture (Evans et al, 2021;Kietzmann, Spoerer, et al, 2019) and learning rules (Zhuang et al, 2021) offer new perspectives for meeting this goal. Such models have yielded important insight in how the brain solves the general problem of object recognition and show improved generalization in some cases (Geirhos, Narayanappa, et al, 2020;Spoerer et al, 2017) but are still outmatched by humans (Geirhos, Meding, et al, 2020;Geirhos, Narayanappa, et al, 2020).…”

Unsupervised changes in core object recognition behavior are predicted by neural plasticity in inferior temporal cortex

“…A fourth future direction is to develop new unsupervised learning algorithms that implement some of the core ideas of temporal contiguity learning, but are scaled to produce high-performing visual systems that are competitive with state-of-the-art neural network systems trained by full supervision. Many computational efforts have touched on this direction ( Agrawal et al, 2015 ; Bahroun and Soltoggio, 2017 ; Goroshin et al, 2014 ; Higgins et al, 2016 ; Kheradpisheh et al, 2016 ; Lotter et al, 2016 ; Srivastava et al, 2015 ; Wang and Gupta, 2015 ; Whitney et al, 2016 ), and some are just beginning to make predictions about the responses along the ventral stream ( Zhuang et al, 2021 ). A key next step will be to put those full-scale models to experimental test at both the neurophysiological and behavioral levels.…”

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

“…VisNet also shows how its type of learning can be performed without prejudging what is to be learned, and without providing a biologically implausible teacher for what the outputs of each neuron should be, which in contrast is assumed in HMAX and deep learning. Indeed, in deep learning with convolution networks the focus is still to categorize based on image properties (Rajalingham et al, 2018;Zhuang et al, 2021), rather than object properties that are revealed for example when objects transform in the world (Rolls, 2021a).…”

“…Although progress has been made in unsupervised versions of deep convolutional neural networks trained with backpropagation of error ( Zhuang et al, 2021 ), the network still relies on image features to discriminate objects, and therefore will have problems with learning view invariant object representations to solve problems such as that illustrated in Figure 7 in which different views of an object have different image properties ( Robinson and Rolls, 2015 ). VisNet solves this and other aspects of invariant object recognition by using the statistics of the world captured by slow learning ( Robinson and Rolls, 2015 ).…”