Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders

Abstract

Category-selectivity in the brain describes the observation that certain spatially localized areas of the cerebral cortex tend to respond robustly and selectively to stimuli from specific limited categories. One of the most well known examples of category-selectivity is the Fusiform Face Area (FFA), an area of the inferior temporal cortex in primates which responds preferentially to images of faces when compared with objects or other generic stimuli. In this work, we leverage the newly introduced Topographic Variational Autoencoder to model of the emergence of such localized category-selectivity in an unsupervised manner. Experimentally, we demonstrate our model yields spatially dense neural clusters selective to faces, bodies, and places through visualized maps of Cohen's d metric. We compare our model with related supervised approaches, namely the TDANN, and discuss both theoretical and empirical similarities. Finally, we show preliminary results suggesting that our model yields a nested spatial hierarchy of increasingly abstract categories, analogous to observations from the human ventral temporal cortex.

Visual object topographic motifs emerge from self-organization of a unified representational space

“…How does the approach taken here relate to concurrently developed techniques bringing spatialized responses to deep neural networks (Lee et al, 2020; Blauch et al, 2022; Keller et al, 2021; Zhang et al, 2021)? Across the set of approaches, all seem to be conceiving of the problem at different levels of abstraction, and test for different signatures.…”

“…In contrast, we cast the problem of topography as one of data-manifold mapping, which is more closely related to the approaches taken by Keller et al, 2021 and Zhang et al, 2021. Keller et al, 2021 trained a topographic variational autoencoder which, like our SOM, was also trained on from the features from of a pre-trained Alexnet model (though appended after the final convolutional stage). This topographic layer is also a grid of units (though, with a circular topology), initialized into the deep net feature space, and trained to maximize the data likelihood using an algorithm related to independent component analysis.…”

“…Both these approaches probe the resulting tuned map with some of the same stimulus sets as in the present work, though we all used different analysis methods to quantify the spatial organization, resulting in some differences (e.g. both Keller et al, 2021 and Zhang et al, 2021 report the presence of body-selective regions). As a whole, these methods use a topographic layer to reveal the untangled data manifold of a pre-trained feature space, rather than to constrain the learning of the features themselves.…”

“…In contrast, we cast the problem of topography as one of data-manifold mapping, which is more closely related to the approaches taken by Keller et al, 2021 and Zhang et al, 2021. Keller et al, 2021 trained a topographic variational autoencoder which, like our SOM, was also trained on from the features from of a pre-trained Alexnet model (though appended after the final convolutional stage).…”

“…One critical missing component of this theoretical account, though, is how to bridge from the multi-dimensional representational spaces of deep neural networks to the spatialized tuning of the cortical sheet—that is, to have a computational account of not only what the tuning is, but also where it is located on a two-dimensional surface. Concurrently, a variety of approaches are emerging to bring spatial organization in deep neural networks, all of which operate at different levels of abstraction regarding the underlying mechanisms (Lee et al, 2020; Blauch et al, 2022; Zhang et al, 2021; Keller et al, 2021). Here, we cast the problem of topography as one of data-manifold mapping, leveraging Kohonen self-organizing maps (Kohonen, 1990).…”