Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing

Kriegeskorte, Nikolaus

doi:10.1146/annurev-vision-082114-035447

Cited by 926 publications

(789 citation statements)

References 82 publications

Supporting

Mentioning

765

Contrasting

Unclassified

Order By: Relevance

“…As others (Kriegeskorte, 2015;Yamins & DiCarlo, 2016), we therefore believe that there is a strong case that DNNs can serve as a model for information processing in the brain. From this perspective, using DNNs to understand the human brain and behavior is similar to using an animal model.…”

Section: 1supporting

confidence: 52%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

Deep learning Cost functions RepresentationsVisual processing a b s t r a c t Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural networks. Machine learning has shown that tasks become easier to solve when they are decomposed into subtasks with their own cost function. We hypothesize that the visual system optimizes multiple cost functions of unrelated tasks and this causes the emergence of a ventral pathway dedicated to vision for perception, and a dorsal pathway dedicated to vision for action.To evaluate the functional organization in multi-task deep neural networks, we propose a method that measures the contribution of a unit towards each task, applying it to two networks that have been trained on either two related or two unrelated tasks, using an identical stimulus set. Results show that the network trained on the unrelated tasks shows a decreasing degree of feature representation sharing towards higher-tier layers while the network trained on related tasks uniformly shows high degree of sharing.We conjecture that the method we propose can be used to analyze the anatomical and functional organization of the visual system and beyond. We predict that the degree to which tasks are related is a good descriptor of the degree to which they share downstream cortical-units.

show abstract

Section: 1supporting

confidence: 52%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

show abstract

“…According to some theories, feedforward functional units perform a common computation: take the afferent input, apply a nonlinearity, adjust the synaptic weights that are being linearly summed, and normalize by the aggregate activity of a nearby neural pool (DiCarlo et al, 2012; Riesenhuber and Poggio, 1999). Repetitive computation throughout multiple layers is also the essence of deep convolutional neural networks, which have recently proven useful in visual neuroscience (Kriegeskorte, 2015; Krizhevsky et al, 2012; Yamins et al, 2014). The architecture consists of repetitive local operations like nonlinear thresholding, normalization, max-pooling, and convolution until a final decision is made.…”

Section: Serial and Repeated Micro-computations That Can Produce Choicementioning

confidence: 99%

Economic Choice as an Untangling of Options into Actions

Yoo

Hayden

2018

Neuron

115

123

View full text Add to dashboard Cite

SUMMARY We propose that economic choice can be understood as a gradual transformation from domain of options to one of the actions. We draw an analogy with the idea of untangling information in the form vision system and propose that form vision and economic choice may be two aspects of a larger process that sculpts actions based on sensory inputs. From this viewpoint, choice results from the accumulated effect of repetitions of simple computations. These may consist primarily of relative valuations (evaluations relative to the value of rejection, perhaps in a manner akin to divisive normalization) applied to individual offers. With regard to economic choice, cortical brain regions differ primarily in their position and in what information they prioritize, and do not—with a few exceptions—have categorically distinct roles. Each region’s specific contribution is determined largely by its inputs; thus, understanding connectivity is crucial for understanding choice. This view suggests that there is no single site of choice, that there is no meaningful distinction between pre- and post-decisionality, and that there is no explicit representation of value in the brain.

show abstract

“…The simplest hypothesis is that each successive stage of parallel processing takes the inputs from the previous stage and applies, in a feedforward fashion, rules of combination that exploit the prior knowledge. Recent computer-vision studies of object recognition in natural scenes show that such feedforward architectures (i.e., deep convolutional neural networks) can reach or exceed human performance in specific domains (see Kriegeskorte 2015). …”

Section: Conceptual Framework For Studying the Link Between V1 Resmentioning

confidence: 99%

Linking V1 Activity to Behavior

Seidemann

Geisler

2018

Annu. Rev. Vis. Sci.

View full text Add to dashboard Cite

A long-term goal of visual neuroscience is to develop and test quantitative models that account for the moment-by-moment relationship between neural responses in early visual cortex and human performance in natural visual tasks. This review focuses on efforts to address this goal by measuring and perturbing the activity of primary visual cortex (V1) neurons while non- human primates perform demanding, well-controlled visual tasks. We start by describing a conceptual approach—the decoder linking model (DLM) framework—in which candidate decoding models take neural responses as input and generate predicted behavior as output. The ultimate goal in this framework is to find the actual decoder—the model that best predicts behavior from neural responses. We discuss key relevant properties of primate V1 and review current literature from the DLM perspective. We conclude by discussing major technological and theoretical advances that are likely to accelerate our understanding of the link between V1 activity and behavior.

show abstract

Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing

Cited by 926 publications

References 82 publications

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Economic Choice as an Untangling of Options into Actions

Linking V1 Activity to Behavior

Contact Info

Product

Resources

About