Evidence accumulation for value computation in the prefrontal cortex during decision making

Abstract

A key step of decision making is to determine the value associated with each option. The evaluation process often depends on the accumulation of evidence from multiple sources, which may arrive at different times. How evidence is accumulated for value computation in the brain during decision making has not been well studied. To address this problem, we trained rhesus monkeys to perform a decision-making task in which they had to make eye movement choices between two targets, whose reward probabilities had to be determined with the combined evidence from four sequentially presented visual stimuli. We studied the encoding of the reward probabilities associated with the stimuli and the eye movements in the orbitofrontal (OFC) and the dorsolateral prefrontal (DLPFC) cortices during the decision process. We found that the OFC neurons encoded the reward probability associated with individual pieces of evidence in the stimulus domain. Importantly, the representation of the reward probability in the OFC was transient, and the OFC did not encode the reward probability associated with the combined evidence from multiple stimuli. The computation of the combined reward probabilities was observed only in the DLPFC and only in the action domain. Furthermore, the reward probability encoding in the DLPFC exhibited an asymmetric pattern of mixed selectivity that supported the computation of the stimulus-to-action transition of reward information. Our results reveal that the OFC and the DLPFC play distinct roles in the value computation during evidence accumulation.

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice

“…Once learned, the mapping between these cues and reward attributes is fixed, and the subject must choose between options with explicitly cued values. Neurons in the central-lateral primate OFC and rat lOFC have been shown to represent the values associated with these sensory cues in their firing rates (although in rats these representations tend to occur after the choice, if the task requires one), as well as reward outcomes and outcome values [7, 29, 33, 34, 40, 48–54]. However, the extent to which OFC is causally required for these tasks is a point of contention, and may differ across species [36, 55, 56].…”

“…However, recordings from rat lOFC in sensory preconditioning paradigms have revealed cue-evoked responses that may reflect inferred value [29,37]. Studies in non-human primates have also reported strong encoding of offer values in OFC after presentation of a stimulus, and before choice [26,[38][39][40]. A recent study in mouse OFC used olfactory cues to convey reward attributes (juice identity and volume) and reported encoding of offer values before choice [41].…”

Signal neutrality, scalar property, and collapsing boundaries as consequences of a learned multi-timescale strategy

“…With this in mind, and given the evidence of reinforcement learning-compatible signaling in the brain [56,57], we advance the hypothesis that the similarities between the model workings and the experimental results originate from a common high-level 'optimal' strategy for dealing with a volatile environment, discovered through the interaction with the rules of the task. matches qualitatively the optimal strategy for fixed-duration tasks [8,10].…”

“…With this in mind, and given the evidence of reinforcement learning-compatible signaling in the brain [56, 57], we advance the hypothesis that the similarities between the model workings and the experimental results originate from a common high-level ‘optimal’ strategy for dealing with a volatile environment, discovered through the interaction with the rules of the task. Though the implementation substrates are very different and present very different degrees of complexity, this shared strategy can help shed light on general information-processing principles leveraged by the brain itself.…”

Bottom-up but not Top-down Attention Dominates the Value Representation in the Orbitofrontal Cortex

“…Our results imply that the OFC does not integrate information from multiple simultaneously presented stimuli, as we showed that the less salient stimuli are ignored by the OFC. Although the representation of the salient value may be interpreted as a decision signal in the OFC, it has been demonstrated that the OFC neurons do not accumulate information from sequentially presented stimuli in a value-based decisionmaking task (Lin et al, 2020). The lack of information integration across space or time suggests that the OFC's role in value processing is more analogous to that of the early visual cortices in visual processing.…”