A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy

Kell, Alexander J.E.; Yamins, Daniel; Shook, Erica N.; Norman-Haignere, Sam V; McDermott, Josh H.

doi:10.1016/j.neuron.2018.03.044

Cited by 471 publications

(658 citation statements)

References 72 publications

Supporting

Mentioning

578

Contrasting

Unclassified

Order By: Relevance

“…It is only at the final, higher-level stages, that the two systems diverge to a specialized system that is finely tuned to facial features, and to a general object system that processes faces and objects similarly, but cannot support face recognition, similar to the primate visual system. A similar divergence at higher layers of processing was recently discovered for auditory stimuli between processing of speech and music in a task-optimized DCNN that generated humanlike performance (14).…”

Section: Discussionsupporting

confidence: 64%

Face recognition depends on specialized mechanisms tuned to view-invariant facial features: Insights from deep neural networks optimized for face or object recognition

Abudarham

Yovel

2020

Preprint

View full text Add to dashboard Cite

SummaryFaces are processed by specialized neural mechanisms in high-level visual cortex. How does this divergence to a face-specific and an object-general system contribute to face recognition? Recent advances in machine face recognition together with our understanding of how humans recognize faces enable us to address this fundamental question. We hypothesized that face recognition depends on a face-selective system that is tuned to view-invariant facial features, which cannot be accomplished by an object-general system. To test this hypothesis, we used deep convolutional neural networks (DCNNs) that were optimized for face or object recognition. Consistent with the hierarchical architecture of the visual system, results show that a human-like, view-invariant face representation emerges only at higher layers of a face-trained but not the object-trained neural network. Thus, by combining human psychophysics and computational modelling of the visual system, we revealed how the functional organization of the visual cortex may contribute to recognition.

show abstract

Section: Discussionsupporting

confidence: 64%

Face recognition depends on specialized mechanisms tuned to view-invariant facial features: Insights from deep neural networks optimized for face or object recognition

Abudarham

Yovel

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…First, deep learning models have been used to simulate actual brain mechanisms, such as in vision (Khaligh-Razavi and Kriegeskorte, 2014;Yamins et al, 2014;Eickenberg et al, 2017) and auditory perception (Kell et al, 2018). Second, DNNs have been applied as tools to analyze neuroscience data, including lesion and tumor segmentation (Pinto et al, 2016;Havaei et al, 2017;Kamnitsas et al, 2017b;, anatomical segmentation (Wachinger et al, 2018;X.…”

Section: Introductionmentioning

confidence: 99%

Deep Neural Networks and Kernel Regression Achieve Comparable Accuracies for Functional Connectivity Prediction of Behavior and Demographics

Kong

Holmes

et al. 2018

Preprint

116

View full text Add to dashboard Cite

There is significant interest in the development and application of deep neural networks (DNNs) to neuroimaging data. A growing literature suggests that DNNs outperform their classical counterparts in a variety of neuroimaging applications, yet there are few direct comparisons of relative utility. Here, we compared the performance of three DNN architectures and a classical machine learning algorithm (kernel regression) in predicting individual phenotypes from whole-brain resting-state functional connectivity (RSFC) patterns. One of the DNNs was a generic fully-connected feedforward neural network, while the other two DNNs were recently published approaches specifically designed to exploit the structure of connectome data. By using a combined sample of almost 10,000 participants from the Human Connectome Project (HCP) and UK Biobank, we showed that the three DNNs and kernel regression achieved similar performance across a wide range of behavioral and demographic measures. Furthermore, the generic feedforward neural network exhibited similar performance to the two state-of-the-art connectome-specific DNNs. When predicting fluid intelligence in the UK Biobank, performance of all algorithms dramatically improved when sample size increased from 100 to 1000 subjects. Improvement was smaller, but still significant, when sample size increased from 1000 to 5000 subjects. Importantly, kernel regression was competitive across all sample sizes. Overall, our study suggests that kernel regression is as effective as DNNs for RSFC-based behavioral prediction, while incurring significantly lower computational costs. Therefore, kernel regression might serve as a useful baseline algorithm for future studies.

show abstract

“…1 However, when deep or recurrent neural networks are used to model neurobiological systems, the comparison between model activity and brain activity is often only verified at a coarse resolution, at the level of entire population dynamics 2,3 , or linear combinations of neurons [4][5][6] , and in contexts that are not very different from the contexts that the networks were originally trained in. Thus, the advent of deep learning as a modeling approach in neuroscience raises two more fundamental unanswered questions.…”

mentioning

confidence: 99%

The dynamic neural code of the retina for natural scenes

Maheswaranathan

Tanaka

et al. 2018

Preprint

View full text Add to dashboard Cite

The normal function of the retina is to convey information about natural visual images. It is this visual environment that has driven evolution, and that is clinically relevant. Yet nearly all of our understanding of the neural computations, biological function, and circuit mechanisms of the retina comes in the context of artificially structured stimuli such as flashing spots, moving bars and white noise. It is fundamentally unclear how these artificial stimuli are related to circuit processes engaged under natural stimuli. A key barrier is the lack of methods for analyzing retinal responses to natural images. We addressed both these issues by applying convolutional neural network models (CNNs) to capture retinal responses to natural scenes. We find that CNN models predict natural scene responses with high accuracy, achieving performance close to the fundamental limits of predictability set by intrinsic cellular variability. Furthermore, individual internal units of the model are highly correlated with actual retinal interneuron responses that were recorded separately and never presented to the model during training. Finally, we find that models fit only to natural scenes, but not white noise, reproduce a range of phenomena previously described using distinct artificial stimuli, including frequency doubling, latency encoding, motion anticipation, fast contrast adaptation, synchronized responses to motion reversal and object motion sensitivity. Further examination of the model revealed extremely rapid context dependence of retinal feature sensitivity under natural scenes using an analysis not feasible from direct examination of retinal responses. Overall, these results show that nonlinear retinal processes engaged by artificial stimuli are also engaged in and relevant to natural visual processing, and that CNN models form a powerful and unifying tool to study how sensory circuitry produces computations in a natural context.

show abstract

A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy

Cited by 471 publications

References 72 publications

Face recognition depends on specialized mechanisms tuned to view-invariant facial features: Insights from deep neural networks optimized for face or object recognition

Face recognition depends on specialized mechanisms tuned to view-invariant facial features: Insights from deep neural networks optimized for face or object recognition

Deep Neural Networks and Kernel Regression Achieve Comparable Accuracies for Functional Connectivity Prediction of Behavior and Demographics

The dynamic neural code of the retina for natural scenes

Contact Info

Product

Resources

About