Rate Dependence and Token Reinforcement? A Preliminary Analysis

Falligant, John Michael; Kranak, Michael P.

doi:10.1007/s40732-022-00521-1

Cited by 2 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These tasks can be modeled with Rescorla–Wagner (RW)-RL models ( Recorla and Wagner, 1972 ), because the choices lead probabilistically, but immediately, to a primary reinforcer ( Bartolo and Averbeck, 2020 ; Beron et al, 2022 ). However, one can also use symbolic reinforcers, for example, tokens or money, to drive learning ( Jackson and Hackenberg, 1996 ; Kirsch et al, 2003 ; Seo and Lee, 2009 ; Delgado et al, 2011 ; Taswell et al, 2018 , 2021 , 2023 ; Falligant and Kranak, 2022 ; Yang et al, 2022 ). In these tasks, subjects learn to make choices to obtain tokens, which are predictive of primary reinforcers in the future.…”

Section: Introductionmentioning

confidence: 99%

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

Burk,

Taswell,

Tang

et al. 2024

J. Neurosci.

View full text Add to dashboard Cite

Reinforcement learning (RL) is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g. money, points) that are later exchanged for primary reinforcers (e.g. food, drink). Although symbolic reinforcers are ubiquitous in our daily lives, widely used in laboratory tasks because they can be motivating, mechanisms by which they become motivating are less understood. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g. current number of accumulated tokens, choice options, task epoch, trials since last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to then correlate with the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n=5 monkeys, three male and two female). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.Significance statementSymbolic reinforcers, like money and points, play a critical role in our lives. However, we lack a mechanistic understanding of how symbolic reinforcement can be related to fluctuations in motivation. We investigated the effect of symbolic reinforcers on behaviors related to motivation during a token reinforcement learning task, using a novel reinforcement learning model and data from five monkeys. We designed a state-based model that can capture reward-predicting features and produce state values at sub-trial resolution. Our findings suggest that the value of a task state can affect willingness to initiate a trial, speed to choose, and persistence to complete a trial. Our model makes testable predictions for within trial fluctuations of neural activity related to values of task states.

show abstract

Section: Introductionmentioning

confidence: 99%

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

Burk,

Taswell,

Tang

et al. 2024

J. Neurosci.

View full text Add to dashboard Cite

show abstract

“…These tasks can be modeled with Rescorla-Wagner (RW) RL models (Recorla, 1972), because the choices lead probabilistically, but immediately, to a primary reinforcer (Bartolo and Averbeck, 2020; Beron et al, 2022). However, one can also use symbolic reinforcers, for example, tokens or money to drive learning (Jackson, 1996; Kirsch et al, 2003; Seo and Lee, 2009; Delgado, Jou and Phelps, 2011; Taswell et al, 2018; Taswell et al, 2021; Falligant and Kranak, 2022; Yang, Li and Stuphorn, 2022; Taswell et al, 2023). In these tasks, subjects learn to make choices to obtain tokens, which can be exchanged in the future for primary reinforcers.…”

Section: Introductionmentioning

confidence: 99%

“…The copyright holder for this preprint this version posted October 11, 2023. ; https://doi.org/10.1101/2023.10.11.561900 doi: bioRxiv preprint 2009; Delgado, Jou and Phelps, 2011;Taswell et al, 2018;Taswell et al, 2021;Falligant and Kranak, 2022;Yang, Li and Stuphorn, 2022;Taswell et al, 2023). In these tasks, subjects learn to make choices to obtain tokens, which can be exchanged in the future for primary reinforcers.…”

Section: Introductionmentioning

confidence: 99%

Computational mechanisms underlying motivation to earn symbolic reinforcers

Burk,

Taswell,

Tang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Rate Dependence and Token Reinforcement? A Preliminary Analysis

Cited by 2 publications

References 37 publications

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

Computational mechanisms underlying motivation to earn symbolic reinforcers

Contact Info

Product

Resources

About