Neural Models of Bayesian Belief Propagation

Rao, Rajesh P. N.

doi:10.7551/mitpress/9780262042383.003.0011

Cited by 16 publications

(16 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Likewise, much work in computational neuroscience focuses on the implementational level, but is Bayesian in character (e.g., Pouget, Dayan, & Zemel, 2003;T. Lee & Mumford, 2003;Zemel, Huys, Natarajan, & Dayan, 2005;Ma, Beck, Latham, & Pouget, 2006;Doya, Ishii, Pouget, & Rao, 2007;Rao, 2007). We discuss the implications of this work in the next section.…”

Section: Optimality: What Does It Mean?mentioning

confidence: 99%

“…Spiking neurons can be modelled as Bayesian integrators accumulating evidence over time (Deneve, 2004;Zemel et al, 2005). Recurrent neural circuits are capable of performing both hierarchical and sequential Bayesian inference (Deneve, 2004;Rao, 2004Rao, , 2007. Even specific brain areas have been studied: for instance, there is evidence that the recurrent loops in the visual cortex integrate top-down priors and bottom-up data in such a way as to implement hierarchical Bayesian inference (T. Lee & Mumford, 2003).…”

Section: Biological Plausibilitymentioning

confidence: 99%

See 1 more Smart Citation

A tutorial introduction to Bayesian models of cognitive development

et al. 2011

View full text Add to dashboard Cite

We present an introduction to Bayesian inference as it is used in probabilistic models of cognitive development. Our goal is to provide an intuitive and accessible guide to the what, the how, and the why of the Bayesian approach: what sorts of problems and data the framework is most relevant for, and how and why it may be useful for developmentalists. We emphasize a qualitative understanding of Bayesian inference, but also include information about additional resources for those interested in the cognitive science applications, mathematical foundations, or machine learning details in more depth. In addition, we discuss some important interpretation issues that often arise when evaluating Bayesian models in cognitive science.

show abstract

Section: Optimality: What Does It Mean?mentioning

confidence: 99%

Section: Biological Plausibilitymentioning

confidence: 99%

A tutorial introduction to Bayesian models of cognitive development

et al. 2011

View full text Add to dashboard Cite

show abstract

“…According to current evidence-integration models (see Bogacz, Brown, Moehlis, Holmes, & Cohen, 2006), in any interval during the decision process, having made the series of observations y and a new observation y new , the decision-maker updates a representation of the posterior probabilities p ( x | y ) by combining them with the likelihoods p ( y new | x ): p ( x | y , y new ) ∝ p ( y new | x ) p ( x | y ). In so-called random walk or drift-diffusion models of two-alternative forced choice decision (Figure 4, top), accumulated evidence is represented in the form of a log posterior ratio, to which is added a log-likelihood ratio representing the evidence from each new observation (see Beck & Pouget, 2007; Bogacz, et al, 2006; Gold & Shadlen, 2007; Rao, 2006; Ratcliff & McKoon, 2008). Given an unlimited number of observations, this procedure is guaranteed to converge to the correct hypothesis.…”

Section: Algorithmic Frameworkmentioning

confidence: 99%

Goal-directed decision making as probabilistic inference: A computational framework and potential neural correlates.

Solway¹,

Botvinick²

2012

Psychological Review

176

185

View full text Add to dashboard Cite

Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology and machine learning, to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and which give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection.

show abstract

“…Several models have been proposed for neural implementation of Bayesian inference (see Rao, 2007 for a review). We focus here on one potential implementation.…”

Section: Modelmentioning

confidence: 99%

Decision Making Under Uncertainty: A Neural Model Based on Partially Observable Markov Decision Processes

Rao¹

2010

Front. Comput. Neurosci.

Self Cite

169

219

View full text Add to dashboard Cite

A fundamental problem faced by animals is learning to select actions based on noisy sensory information and incomplete knowledge of the world. It has been suggested that the brain engages in Bayesian inference during perception but how such probabilistic representations are used to select actions has remained unclear. Here we propose a neural model of action selection and decision making based on the theory of partially observable Markov decision processes (POMDPs). Actions are selected based not on a single “optimal” estimate of state but on the posterior distribution over states (the “belief” state). We show how such a model provides a unified framework for explaining experimental results in decision making that involve both information gathering and overt actions. The model utilizes temporal difference (TD) learning for maximizing expected reward. The resulting neural architecture posits an active role for the neocortex in belief computation while ascribing a role to the basal ganglia in belief representation, value computation, and action selection. When applied to the random dots motion discrimination task, model neurons representing belief exhibit responses similar to those of LIP neurons in primate neocortex. The appropriate threshold for switching from information gathering to overt actions emerges naturally during reward maximization. Additionally, the time course of reward prediction error in the model shares similarities with dopaminergic responses in the basal ganglia during the random dots task. For tasks with a deadline, the model learns a decision making strategy that changes with elapsed time, predicting a collapsing decision threshold consistent with some experimental studies. The model provides a new framework for understanding neural decision making and suggests an important role for interactions between the neocortex and the basal ganglia in learning the mapping between probabilistic sensory representations and actions that maximize rewards.

show abstract

Neural Models of Bayesian Belief Propagation

Cited by 16 publications

References 34 publications

A tutorial introduction to Bayesian models of cognitive development

A tutorial introduction to Bayesian models of cognitive development

Goal-directed decision making as probabilistic inference: A computational framework and potential neural correlates.

Decision Making Under Uncertainty: A Neural Model Based on Partially Observable Markov Decision Processes

Contact Info

Product

Resources

About