Face recognition depends on specialized mechanisms tuned to view-invariant facial features: Insights from deep neural networks optimized for face or object recognition

Abstract

SummaryFaces are processed by specialized neural mechanisms in high-level visual cortex. How does this divergence to a face-specific and an object-general system contribute to face recognition? Recent advances in machine face recognition together with our understanding of how humans recognize faces enable us to address this fundamental question. We hypothesized that face recognition depends on a face-selective system that is tuned to view-invariant facial features, which cannot be accomplished by an object-general system. To test this hypothesis, we used deep convolutional neural networks (DCNNs) that were optimized for face or object recognition. Consistent with the hierarchical architecture of the visual system, results show that a human-like, view-invariant face representation emerges only at higher layers of a face-trained but not the object-trained neural network. Thus, by combining human psychophysics and computational modelling of the visual system, we revealed how the functional organization of the visual cortex may contribute to recognition.

Developmental prosopagnosics and super recognizers rely on the same facial features used by individuals with normal face recognition abilities for face identification

“…Taken together these findings indicate that the ability to tune to critical features and overlook non-critical features is important for face identification. The relationship between sensitivity to critical features and performance on a face perception task has been recently further supported in a study that used a face recognition deep convolutional neural network (DCNN) to model human face recognition (Abudarham & Yovel, 2020). Using the same stimuli used in the current study, this study examined the sensitivity of DCNNs to critical features across its different layers.…”

“…These high-level features enable identification of faces across different head views (Chang & Tsao, 2017, Abudarham et al, 2020.…”

Deep learning algorithms reveal a new visual-semantic representation of familiar faces in human perception and memory

“…A pure semantic DNN increases the total explained variance in human memory to 67%. Overall, these findings show that visualsemantic and semantic DNNs significantly improve the prediction of human representations of familiar faces, beyond the pure visual algorithm that has been so far used to model human face representations 3,4,[6][7][8]10,11,[28][29][30] .…”

“…Face recognition DCNNs show remarkable similarities with human cognitive and neural representations 3,4,[6][7][8]10,11 . These algorithms are trained on face images and therefore exclusively rely on visual information.…”

Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition

“…Fourth, neurophysiological evidence supporting the claim of a progressive computation of view-invariant identity across processing layers is also consistent with data from deep networks. Abudarham and Yovel (2020) , for example, traced the similarity of representations for images of faces across DCNN layers. Earlier layers in the network showed view specificity, whereas higher layers showed view invariance.…”