Optimal utility and probability functions for agents with finite computational precision

Juechems, Keno; Balaguer, Jan; Spitzer, Bernhard; Summerfield, Christopher

doi:10.1073/pnas.2002232118

Cited by 34 publications

(47 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results validate this notion, and imply that the development of AI algorithms that aim to resemble neurobehavioral function should go beyond the objective of maximizing only the accurate transmission of information and account for the motivational aspects of the environment that enable the organism (or the artificial agent) to maximize fitness. Finally, although drawn from a different domain of behavior, our results lend substantial support to economic theories positing that context-dependent utility functions should maximize expected reward rather than the expected accuracy of decisions guided by reward (15,29,35). The corroborating evidence presented in our work grounded on the principles of neural coding and decision behaviour should help to advance the development and refinement of these theories within economics and related disciplines of evolutionary biology and social sciences (36)(37)(38).…”

Section: Discussionsupporting

confidence: 75%

“…Finally, although drawn from a different domain of behavior, our results lend substantial support to economic theories positing that context-dependent utility functions should maximize expected reward rather than the expected accuracy of decisions guided by reward (15, 29, 35). The corroborating evidence presented in our work grounded on the principles of neural coding and decision behaviour should help to advance the development and refinement of these theories within economics and related disciplines of evolutionary biology and social sciences (36–38).…”

Section: Discussionsupporting

confidence: 68%

See 1 more Smart Citation

Neural codes in early sensory areas maximize fitness

Hare

et al. 2021

Preprint

View full text Add to dashboard Cite

It has generally been presumed that sensory information encoded by a nervous system should be as accurate as its biological limitations allow. However, perhaps counter intuitively, accurate representations of sensory signals do not necessarily maximize the organism's chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Initially, we applied predictions of this model to neural responses from large monopolar cells (LMCs) in the blowfly retina. We found that neural codes that maximize reward expectation---and not accurate sensory representations---account for retinal LMC activity. We also conducted experiments in humans and find that early sensory areas flexibly adopt neural codes that promote fitness maximization in a retinotopically-specific manner, which impacted decision behavior. Thus, our results provide evidence that fitness-maximizing rules imposed by the environment are applied at the earliest stages of sensory processing.

show abstract

Section: Discussionsupporting

confidence: 75%

Section: Discussionsupporting

confidence: 68%

Neural codes in early sensory areas maximize fitness

Hare

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…27,43,44 ). Unlike with optimal cognitive biases reported previously [45][46][47][48][49] , we did not find the benefit of the present learning asymmetries to emerge from general limitations (noise) in decision making (supplementary Fig. S4).…”

Section: Discussioncontrasting

confidence: 99%

Asymmetric learning facilitates human inference of transitive relations

Ciranka

Linde-Domingo

Padezhki

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Humans and other animals are capable of inferring never-experienced relations (e.g., A>C) from other relational observations (e.g., A>B and B>C). The processes behind such transitive inference are subject to intense research. Here, we demonstrate a new aspect of relational learning, building on previous evidence that transitive inference can be accomplished through simple reinforcement learning mechanisms. We show in simulations that inference of novel relations benefits from an asymmetric learning policy, where observers update only their belief about the winner (or loser) in a pair. Across 4 experiments (n=145), we find substantial empirical support for such asymmetries in inferential learning. The learning policy favoured by our simulations and experiments gives rise to a compression of values which is routinely observed in psychophysics and behavioural economics. In other words, a seemingly biased learning strategy that yields well-known cognitive distortions can be beneficial for transitive inferential judgments.

show abstract

“…For the Un-crossed condition, we saw that a minority of participants also learned absolute-like codes, and some in the Crossed condition persisted with relative encoding -even though it was clearly sub-optimal. Such individual differences may be driven by cognitive capacity limitations 34 , by intrinsic computational noise 35,36 , by mechanisms relating to working memory or attention 37,38 , or the belief that item-specific value encoding was relevant (despite the absence of such instruction). Future work might manipulate task demands, or measure cognitive capacity, to address these questions.…”

Section: Discussionmentioning

confidence: 99%

Human value learning and representation reflects rational adaptation to task demands

Juechems¹,

Altun²,

Hira³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

When making decisions about goods and actions, humans and animals often rely on internally represented values. However, to be useful across a wide range of contexts, these values need to be represented on an absolute scale – a coding scheme that is computationally costly. By contrast, representing values in a way that depends entirely on context is highly computationally efficient, but can lead to irrational behaviour when values need to be compared across contexts. Thus, an efficient learner would allocate limited computational resources only when needed according to their expectations about the future. Here, we test the hypothesis that value representation is not fixed, but rationally adapted to expectations in two human learning experiments. Unlike most lab-based tasks, participants could use their initial experience (Phase 1) to optimise behaviour (Phase 2). Phase 1 was designed to cause one group to expect only decisions within local contexts (relative code sufficient), and another group to expect choices across local contexts (relative code insufficient). Despite identical learning experiences, the group whose expectations included choices across local contexts, went on to learn absolute value representations, and learned more absolute-like representations than the other group. Human value representation is neither relative nor absolute, but efficiently and rationally tuned to task demands.

show abstract

Optimal utility and probability functions for agents with finite computational precision

Cited by 34 publications

References 55 publications

Neural codes in early sensory areas maximize fitness

Neural codes in early sensory areas maximize fitness

Asymmetric learning facilitates human inference of transitive relations

Human value learning and representation reflects rational adaptation to task demands

Contact Info

Product

Resources

About