The learning of prospective and retrospective cognitive maps within neural circuits

Namboodiri, Vijay Mohan K.; Stuber, Garret D.

doi:10.1016/j.neuron.2021.09.034

Cited by 25 publications

(19 citation statements)

References 231 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative approach to learning cue-reward associations is to infer the cause of meaningful outcomes such as rewards ( 18 – 20 ) (Fig. 1, A and C).…”

mentioning

confidence: 99%

Mesolimbic dopamine release conveys causal associations

Jeong

Taylor

Floeder

et al. 2022

Science

122

133

View full text Add to dashboard Cite

Learning to predict rewards based on environmental cues is essential for survival. It is believed that animals learn to predict rewards by updating predictions whenever the outcome deviates from expectations, and that such reward prediction errors (RPEs) are signaled by the mesolimbic dopamine system—a key controller of learning. However, instead of learning prospective predictions from RPEs, animals can infer predictions by learning the retrospective cause of rewards. Hence, whether mesolimbic dopamine instead conveys a causal associative signal that sometimes resembles RPE remains unknown. We developed an algorithm for retrospective causal learning and found that mesolimbic dopamine release conveys causal associations but not RPE, thereby challenging the dominant theory of reward learning. Our results reshape the conceptual and biological framework for associative learning.

show abstract

“…An alternative approach to learning cue-reward associations is to infer the cause of meaningful outcomes such as rewards ( 18 – 20 ) (Fig. 1, A and C).…”

mentioning

confidence: 99%

Mesolimbic dopamine release conveys causal associations

Jeong

Taylor

Floeder

et al. 2022

Science

122

133

View full text Add to dashboard Cite

show abstract

“…The learning rate is a scalar function of this ratio; the greater the ratio, the faster subjects learn over most of its range, until the ratio approaches 1, the value at which the CS provides no information about the rate of reinforcement, at which point the rate of learning becomes 0 ( Gibbon and Balsam, 1981 , Figure 7 .2, p. 224). These ratios and intervals would appear to be extracted from remembered experience by arithmetic operations on temporal maps and remembered durations ( Honig, 1981 ; Taylor et al, 2014 ; Chandran and Thorwart, 2021 ; Namboodiri and Stuber, 2021 ).…”

Section: Discussionmentioning

confidence: 99%

Quantitative properties of the creation and activation of a cell-intrinsic duration-encoding engram

Gallistel

Johansson²,

Jirenhed³

et al. 2022

Front. Comput. Neurosci.

View full text Add to dashboard Cite

The engram encoding the interval between the conditional stimulus (CS) and the unconditional stimulus (US) in eyeblink conditioning resides within a small population of cerebellar Purkinje cells. CSs activate this engram to produce a pause in the spontaneous firing rate of the cell, which times the CS-conditional blink. We developed a Bayesian algorithm that finds pause onsets and offsets in the records from individual CS-alone trials. We find that the pause consists of a single unusually long interspike interval. Its onset and offset latencies and their trial-to-trial variability are proportional to the CS-US interval. The coefficient of variation (CoV = σ/μ) are comparable to the CoVs for the conditional eye blink. The average trial-to-trial correlation between the onset latencies and the offset latencies is close to 0, implying that the onsets and offsets are mediated by two stochastically independent readings of the engram. The onset of the pause is step-like; there is no decline in firing rate between the onset of the CS and the onset of the pause. A single presynaptic spike volley suffices to trigger the reading of the engram; and the pause parameters are unaffected by subsequent volleys. The Fano factors for trial-to-trial variations in the distribution of interspike intervals within the intertrial intervals indicate pronounced non-stationarity in the endogenous spontaneous spiking rate, on which the CS-triggered firing pause supervenes. These properties of the spontaneous firing and of the engram read out may prove useful in finding the cell-intrinsic, molecular-level structure that encodes the CS-US interval.

show abstract

“…Under the retrospective strategy, learning starts from the rewarded state and animals learn about how the rewarded state is predicted by preceding behavioral events. Recent studies support that animals do learn about retrospective associative relationships between behavioral events 17 , and the retrospective strategy might be especially powerful during the learning process for discovering the connection between rare rewards with preceding behavioral events 15 . The current study seeks to provide empirical support for the use of retrospective strategy during the learning process.…”

Section: Introductionmentioning

confidence: 99%

“…The learning strategies for building the internal model can be conceptualized under two broad categories: prospective 8,9 and retrospective strategies 15,16 . Under the prospective strategy, animals learn about how the current behavioral state predicts events and rewards in the future.…”

Section: Introductionmentioning

confidence: 99%

“…This forward-looking prospective strategy is widely used in well-learned contexts as an effective means to guide decision making through the prediction of future rewards. An alternative, but not mutually exclusive, strategy that has been proposed recently is the backward-looking retrospective strategy 15,16 . Under the retrospective strategy, learning starts from the rewarded state and animals learn about how the rewarded state is predicted by preceding behavioral events.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The behavioral signature of stepwise learning strategy and its neural correlate in the basal forebrain

Manzur

Vlasov

Lin

2022

Preprint

View full text Add to dashboard Cite

Associative learning is a fundamental cognitive capacity that allows animals and humans to learn the predictive relationship between behavioral events and rewarding outcomes. While the process of learning is commonly conceptualized as a prospective strategy (learning how behavioral events predict future rewards), here we provide behavioral and neurophysiological evidence to show that animals may instead employ a retrospective and stepwise learning strategy (learning how the reward is predicted by preceding behavioral events). In rats learning a new association in which the reward was paired with a sequence of behavioral events, learning started from the event closest to the reward and sequentially incorporated earlier events into animals' internal model. The learning of each behavioral event as a new reward predictor was accompanied by the emergence of basal forebrain (BF) neuronal responses toward that event. BF activities quantitatively conveyed a reward prediction error signal associated with the behavioral event, and promoted reward-seeking behavioral sequences containing the newly learned event. As the internal model incorporated more behavioral events as reward predictors, non-rewarded behavioral sequences that were once compatible with the internal model during early stages of learning became incompatible and were sequentially eliminated. Together, these results demonstrate how the retrospective and stepwise learning strategy can effectively establish animals' internal model during the learning process and lead to the sequential refinement of reward-seeking behaviors. These results also highlight the functional significance of BF neuronal activities, which provided unique insights into the covert dynamics of the learning process in single trials.

show abstract

The learning of prospective and retrospective cognitive maps within neural circuits

Cited by 25 publications

References 231 publications

Mesolimbic dopamine release conveys causal associations

Mesolimbic dopamine release conveys causal associations

Quantitative properties of the creation and activation of a cell-intrinsic duration-encoding engram

The behavioral signature of stepwise learning strategy and its neural correlate in the basal forebrain

Contact Info

Product

Resources

About