If deep learning is the answer, what is the question?

Abstract

Neuroscience research is undergoing a minor revolution. Recent advances in machine learning and artificial intelligence (AI) research have opened up new ways of thinking about neural computation. Many researchers are excited by the possibility that deep neural networks may offer theories of perception, cognition and action for biological brains. This perspective has the potential to radically reshape our approach to understanding neural systems, because the computations performed by deep networks are learned from experience, not endowed by the researcher. If so, how can neuroscientists use deep networks to model and understand biological brains? What is the outlook for neuroscientists who seek to characterise computations or neural codes, or who wish to understand perception, attention, memory, and executive functions? In this Perspective, our goal is to offer a roadmap for systems neuroscience research in the age of deep learning. We discuss the conceptual and methodological challenges of comparing behaviour, learning dynamics, and neural representation in artificial and biological systems. We highlight new research questions that have emerged for neuroscience as a direct consequence of recent advances in machine learning.

The role of mental simulation in primate physical inference abilities

“…Instead, the networks that most successfully capture primate behavior were those that were additionally constrained to perform simulations. This approach may be understood as a generalization of simultaneously optimizing on multiple tasks (Yang, Cole, and Rajan 2019; Yang et al 2019), or of optimizing for specific tasks in the face of specific regularization (Sussillo et al 2015; Lee and DiCarlo 2019), with the goal of building interpretable models of behavioral and neural phenomena (Saxe, Nelli, and Summerfield 2021). To this end, our work highlights a novel and general approach for testing hypotheses about specific inductive biases that govern human cognition by directly comparing models that do or do not implement those biases.…”

Using Deep Learning to Predict Human Decisions, and Cognitive Models to Explain Deep Learning Models

“…It is therefore not surprising that these tools were adopted by cognitive neuroscientists, examining how such models represent information and the extent to which the operation of such models is similar to information representation and processing in the brain [11,13]. However, these models’ level of complexity, and their tendency to track regularities in the data in an unpredictable and opaque manner, means that while they may perform well, their explanatory power is limited [7,12,37]. Fortunately, DNNs can be a useful scientific tool in a number of ways, one of them being for exploratory investigations [38].…”

A connectome of theDrosophilacentral complex reveals network motifs suitable for flexible navigation and context-dependent action selection

“…However, insights into how such artificial neural networks generate the representations observed in their units-something that could, in principle, guide mechanistic hypotheses for the function of natural neural networks-have been slower to come (but see (Cueva et al, 2019;Uria et al, 2020) for progress in uncovering the architectural basis of navigational responses in these networks). In this era of deep learning, a broader question concerns the level of understanding that is appropriate or even possible for the function of large and complex neural networks (Gao and Ganguli, 2015;Hasson et al, 2020;Lillicrap and Kording, 2019;Richards et al, 2019;Saxe et al, 2020;Yamins and DiCarlo, 2016). What seems achievable is an understanding of learning rules and objective functions that can, in principle, generate networks with realistic population responses for specific cognitive tasks.…”