A Bayesian model of context-sensitive value attribution

Abstract

Substantial evidence indicates that incentive value depends on an anticipation of rewards within a given context. However, the computations underlying this context sensitivity remain unknown. To address this question, we introduce a normative (Bayesian) account of how rewards map to incentive values. This assumes that the brain inverts a model of how rewards are generated. Key features of our account include (i) an influence of prior beliefs about the context in which rewards are delivered (weighted by their reliability in a Bayes-optimal fashion), (ii) the notion that incentive values correspond to precision-weighted prediction errors, (iii) and contextual information unfolding at different hierarchical levels. This formulation implies that incentive value is intrinsically context-dependent. We provide empirical support for this model by showing that incentive value is influenced by context variability and by hierarchically nested contexts. The perspective we introduce generates new empirical predictions that might help explaining psychopathologies, such as addiction.DOI: http://dx.doi.org/10.7554/eLife.16127.001

A Goal-Directed Bayesian Framework for Categorization

“…This aspect of our model resonates with the original idea of a categorical hierarchy and sheds new light on its characteristics. The same justification of hierarchical models, advocated here in relation to features, can be generalized to action and goal variables, establishing a link, for instance, with hierarchical reinforcement learning (Barto and Mahadevan, 2003; Botvinick et al, 2009; Rigoli et al, 2016b). Our argument is also relevant for neurobiological theories of hierarchical brain organization (Friston, 2005, 2010; Pezzulo et al, 2015), raising the possibility that the functional segregation might reflect distinct representations of latent causes or clusters of latent causes, which are independent (and processed in parallel) or organized hierarchically (and processed along a neural hierarchy) (Friston and Buzsáki, 2016; Mirza et al, 2016).…”

Reference-dependent preferences arise from structure learning

“…Our main point of departure is the idea that the reference point corresponds to an agent's expectation [2,3,4], rationally updating over time using Bayes' rule and a nonparametric prior that allows the agent to cluster its experience into distinct latent causes [25,19]. The expectation (and thus the reference point) can sometimes "jump" rather than adapt slowly.…”

“…Modern theories of decision making have sought to formalize the concept of an expectation-based reference point and how it changes based on experience. In an influential line of work, Kőszegi and Rabin [2,3] proposed that reference points reflect rational expectations based on recent outcomes (see also [4,5]). In support of this theory, contestants on the TV game show "Deal or No Deal" were more likely to make risky choices when, upon receiving information that suggested they would take home less money than they expected [6], recapitulating results from laboratory experiments [7,8].…”

Better than expected: the influence of option expectations during decision-making

“…This perspective has implications for scenarios where options are offered sequentially. Previous accounts have assumed that all options within a trial are equally affected by the context, even when presentation is sequential [7,8]. On the contrary, the model proposed by Rigoli et al [7,8] implicates that options presented sequentially will not all be equally affected by context.…”

“…This influence would be exerted by expectations about the value of options. The model from which this proposal is derived [7,8] was developed to explain context effects on value attribution and choice. A central premise is that individuals keep track of the distribution of rewards encountered in an environment or context.…”