Goal-directed decision making as probabilistic inference: A computational framework and potential neural correlates.

Abstract

Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology and machine learning, to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and which give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection.

Evidence integration in model-based tree search

“…Despite such advances, however, comparatively little progress has so far been made toward characterizing the concrete process by which model-based decisions are reached, that is, the actual procedure through which a representation of the decision problem is translated into a choice (9,10,23). This situation contrasts sharply with what one finds in the literature on simple choice, where a number of detailed process models have been proposed.…”

“…In the most salient work along these lines, reward-based tree search has been conceptualized in terms provided by modelbased reinforcement learning, a computational framework in which reward-based decisions are based on an explicit model of the choice problem, a "cognitive map" of the decision tree itself (11). Under this rubric, recent work has illuminated several aspects of rewardbased tree search, providing an indication of how representations of decision problems are acquired and updated (12)(13)(14), where in the brain relevant quantities (e.g., cumulative rewards) are represented (15-17), and how model-based decision making interacts with simpler, habit-based choice mechanisms (15,(18)(19)(20)(21)(22).Despite such advances, however, comparatively little progress has so far been made toward characterizing the concrete process by which model-based decisions are reached, that is, the actual procedure through which a representation of the decision problem is translated into a choice (9,10,23). This situation contrasts sharply with what one finds in the literature on simple choice, where a number of detailed process models have been proposed.…”

“Can waiting awaken the resting brain?” A comparison of waiting- and cognitive task-induced attenuation of very low frequency neural oscillations

“…In early formulations relating to the DMN, it was postulated that attenuation of resting neural activity occurred during goal-directed behaviors without reference to the cognitive character of these tasks (Raichle et al, 2001). Goal-directedness refers to an intrinsic drive to achieve certain objectives or desired results (Solway and Botvinick, 2012). In this sense, both 2CRT and waiting tasks could be considered goal-directed even if external stimulation and information processing demands were minimal as was the case in the waiting tasks.…”

A social Bayesian brain: How social knowledge can shape visual perception

“…Minimization of prediction error through action is called 'active inference' . Currently, the active inference model has been applied to basic forms of behaviour, such as oculomotor control and simple goaldirected decision-making (Solway & Botvinick, 2012). With regards to social behaviour, the idea of active inference suggests that people are likely to employ actions to confirm their social preconceptions.…”