Face Recognition Depends on Specialized Mechanisms Tuned to View‐Invariant Facial Features: Insights from Deep Neural Networks Optimized for Face or Object Recognition

Abudarham, Naphtali; Grosbard, Idan

doi:10.1111/cogs.13031

Cited by 43 publications

(25 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such and many other modifications (e.g., multi-modal self-supervised image-language training, in the way DNNs are built and trained may generate perceptual effects that are more human-like (Shoham, Grosbard, Patashnik, Cohen-Or, & Yovel, 2022). Yet even current DNNs can advance our understanding of the nature of the high-level representations that are required for face and object recognition (Abudarham, Grosbard, & Yovel, 2021;Hill et al, 2019), which are still undefined in current neural and cognitive models. This significant computational achievement should not be dismissed.…”

Section: Introductionmentioning

confidence: 99%

Deep problems with neural network models of human vision

Bowers¹,

Dujmović²,

Montero³

et al. 2022

Behav Brain Sci

View full text Add to dashboard Cite

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral datasets, and (3) DNNs do the best job in predicting brain signals in response to images taken from various brain datasets (e.g., single cell responses or fMRI data). However, these behavioral and brain datasets do not test hypotheses regarding what features are contributing to good predictions and we show that the predictions may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychological research. This contradicts the common claim that DNNs are good, let alone the best, models of human object recognition. We argue that theorists interested in developing biologically plausible models of human vision need to direct their attention to explaining psychological findings. More generally, theorists need to build models that explain the results of experiments that manipulate independent variables designed to test hypotheses rather than compete on making the best predictions. We conclude by briefly summarizing various promising modelling approaches that focus on psychological data.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep problems with neural network models of human vision

Bowers¹,

Dujmović²,

Montero³

et al. 2022

Behav Brain Sci

View full text Add to dashboard Cite

show abstract

“…To address these limitations in human studies, in the current study, we used deep convolutional neural networks (DCNNs) as computational models of perceptual expertise. DCNNs are brain-inspired algorithms that reach humanlevel performance and generate human-like representations for objects and faces [31][32][33][34][35][36]. These models can be trained to classify images from different domains at different levels of categorization [23].…”

Section: Introductionmentioning

confidence: 99%

Deep learning models challenge the prevailing assumption that face-like effects for objects of expertise support domain-general mechanisms

2023

Self Cite

View full text Add to dashboard Cite

The question of whether task performance is best achieved by domain-specific, or domain-general processing mechanisms is fundemental for both artificial and biological systems. This question has generated a fierce debate in the study of expert object recognition. Because humans are experts in face recognition, face-like neural and cognitive effects for objects of expertise were considered support for domain-general mechanisms. However, effects of domain, experience and level of categorization, are confounded in human studies, which may lead to erroneous inferences. To overcome these limitations, we trained deep learning algorithms on different domains (objects, faces, birds) and levels of categorization (basic, sub-ordinate, individual), matched for amount of experience. Like humans, the models generated a larger inversion effect for faces than for objects. Importantly, a face-like inversion effect was found for individual-based categorization of non-faces (birds) but only in a network specialized for that domain. Thus, contrary to prevalent assumptions, face-like effects for objects of expertise do not support domain-general mechanisms but may originate from domain-specific mechanisms. More generally, we show how deep learning algorithms can be used to dissociate factors that are inherently confounded in the natural environment of biological organisms to test hypotheses about their isolated contributions to cognition and behaviour.

show abstract

“…Feedforward CNNs remain among the best models for predicting mid- and high-level cortical representations of novel natural images within the first 100-200 ms after stimulus onset [7, 8]. Diverse CNN models, trained on tasks such as face identification [9, 10], object recognition [11], inverse graphics [12], and unsupervised generative modeling [13] have all been shown to replicate at least some aspects of face-patch system representations. Face-selective artificial neurons occur even in untrained CNNs [14], and functional specialization between object and face representation emerges in CNNs trained on the dual task of recognizing objects and identifying faces [15].…”

Section: Introductionmentioning

confidence: 99%

Emergence of brain-like mirror-symmetric viewpoint tuning in convolutional neural networks

Farzmahdi

Zarco

Freiwald

et al. 2023

Preprint

View full text Add to dashboard Cite

Primates can recognize objects despite 3D geometric variations such as in-depth rotations. The computational mechanisms that give rise to such invariances are yet to be fully understood. A curious case of partial invariance occurs in the macaque face-patch AL and in fully connected layers of deep convolutional networks in which neurons respond similarly to mirror-symmetric views (e.g., left and right profiles). Why does this tuning develop? Here, we propose a simple learning-driven explanation for mirror-symmetric viewpoint tuning. We show that mirror-symmetric viewpoint tuning for faces emerges in the fully connected layers of convolutional deep neural networks trained on object recognition tasks, even when the training dataset does not include faces. First, using 3D objects rendered from multiple views as test stimuli, we demonstrate that mirror-symmetric viewpoint tuning in convolutional neural network models is not unique to faces: it emerges for multiple object categories with bilateral symmetry. Second, we show why this invariance emerges in the models. Learning to discriminate among bilaterally symmetric object categories induces reflection-equivariant intermediate representations. AL-like mirror-symmetric tuning is achieved when such equivariant responses are spatially pooled by downstream units with sufficiently large receptive fields. These results explain how mirror-symmetric viewpoint tuning can emerge in neural networks, providing a theory of how they might emerge in the primate brain. Our theory predicts that mirror-symmetric viewpoint tuning can emerge as a consequence of exposure to bilaterally symmetric objects beyond the category of faces, and that it can generalize beyond previously experienced object categories.

show abstract

Face Recognition Depends on Specialized Mechanisms Tuned to View‐Invariant Facial Features: Insights from Deep Neural Networks Optimized for Face or Object Recognition

Cited by 43 publications

References 34 publications

Deep problems with neural network models of human vision

Deep problems with neural network models of human vision

Deep learning models challenge the prevailing assumption that face-like effects for objects of expertise support domain-general mechanisms

Emergence of brain-like mirror-symmetric viewpoint tuning in convolutional neural networks

Contact Info

Product

Resources

About