The Neurobiology of Impulsive Decision-Making and Reinforcement Learning in Nonhuman Animals

Groman, Stephanie M.

doi:10.1007/7854_2020_127

Cited by 13 publications

(8 citation statements)

References 168 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Acetylcholine on a global level in the brain has been found to be relevant in almost every decision-making paradigm. Lesioning studies of the basal forebrain have shown disruptions in reversal learning in marmosets [ 182 ], as well as being implicated in memory storage [ 183 , 184 ]. The role of ACh in memory has been shown to act as a modulator of update speed and as a controller of meta learning [ 185 ].…”

Section: Role Of Acetylcholine In the Brainmentioning

confidence: 99%

The Neuromodulatory Role of the Noradrenergic and Cholinergic Systems and Their Interplay in Cognitive Functions: A Focused Review

et al. 2022

View full text Add to dashboard Cite

The noradrenergic and cholinergic modulation of functionally distinct regions of the brain has become one of the primary organizational principles behind understanding the contribution of each system to the diversity of neural computation in the central nervous system. Decades of work has shown that a diverse family of receptors, stratified across different brain regions, and circuit-specific afferent and efferent projections play a critical role in helping such widespread neuromodulatory systems obtain substantial heterogeneity in neural information processing. This review briefly discusses the anatomical layout of both the noradrenergic and cholinergic systems, as well as the types and distributions of relevant receptors for each system. Previous work characterizing the direct and indirect interaction between these two systems is discussed, especially in the context of higher order cognitive functions such as attention, learning, and the decision-making process. Though a substantial amount of work has been done to characterize the role of each neuromodulator, a cohesive understanding of the region-specific cooperation of these two systems is not yet fully realized. For the field to progress, new experiments will need to be conducted that capitalize on the modular subdivisions of the brain and systematically explore the role of norepinephrine and acetylcholine in each of these subunits and across the full range of receptors expressed in different cell types in these regions.

show abstract

Section: Role Of Acetylcholine In the Brainmentioning

confidence: 99%

The Neuromodulatory Role of the Noradrenergic and Cholinergic Systems and Their Interplay in Cognitive Functions: A Focused Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In proposing this method, we do not wish to discount the large and impressive nonhuman literature on the behavioral neuroscience of impulsive and risky choice (e.g., seeDalley & Robbins, 2017;Fellows, 2004;Groman, 2020;Isles et al, 2019;Madden & Bickel, 2010 for some illustrative reviews and perspectives). Rather, our aim is to facilitate a more effective integration of behavior-analytic method/theory into the behavioral neuroscience of impulsive and risky choice.Journal of the Experimental Analysis of Behavior 2022, 117, 363-383 NUMBER 3 (MAY) © 2022 Society for the Experimental Analysis of Behavior.…”

mentioning

confidence: 99%

A method for studying reinforcement factors controlling impulsive choice for use in behavioral neuroscience

Hughes

Langford

Heukelom

et al. 2022

J Exper Analysis Behavior

View full text Add to dashboard Cite

Although procedures originating within the experimental analysis of behavior commonly are used in behavioral neuroscience to produce behavioral endpoints, they are used less often to analyze the behavioral processes involved, particularly at the level of individual organisms (see Soto, 2020). Concurrent‐chains procedures have been used extensively to study choice and to quantify relations between various dimensions of reinforcement and preference. Unfortunately, parametric analysis of those relations using traditional steady‐state, single‐subject experimental designs can be time‐consuming, often rendering these procedures impractical for use in behavioral neuroscience. The purpose of this paper is to describe how concurrent‐chains procedures can be adapted to allow for parametric examination of effects of the reinforcement dimensions involved in impulsive choice (magnitude and delay) within experimental sessions in rats. Data are presented indicating that this procedure can produce relatively consistent within‐session estimates of sensitivity to reinforcement in individual subjects, and that these estimates can be modified by neurobiological manipulation (drug administration). These data suggest that this type of procedure offers a promising approach to the study of neurobiological mechanisms of complex behavior in individual organisms, which could facilitate a more fruitful relationship between behavior analysis and behavioral neuroscience.

show abstract

“…In summary, previous work suggests that impulsive decision-making in clinical groups is maladaptive [ 94 , 95 ]. In contrast, our results across the three tasks suggest that impulsive behavior is not inherently negative and can be beneficial when an environment is more volatile than expected.…”

Section: Discussionmentioning

confidence: 97%

Environmental uncertainty and the advantage of impulsive choice strategies

Burk

Averbeck

2023

PLoS Comput Biol

View full text Add to dashboard Cite

Choice impulsivity is characterized by the choice of immediate, smaller reward options over future, larger reward options, and is often thought to be associated with negative life outcomes. However, some environments make future rewards more uncertain, and in these environments impulsive choices can be beneficial. Here we examined the conditions under which impulsive vs. non-impulsive decision strategies would be advantageous. We used Markov Decision Processes (MDPs) to model three common decision-making tasks: Temporal Discounting, Information Sampling, and an Explore-Exploit task. We manipulated environmental variables to create circumstances where future outcomes were relatively uncertain. We then manipulated the discount factor of an MDP agent, which affects the value of immediate versus future rewards, to model impulsive and non-impulsive behavior. This allowed us to examine the performance of impulsive and non-impulsive agents in more or less predictable environments. In Temporal Discounting, we manipulated the transition probability to delayed rewards and found that the agent with the lower discount factor (i.e. the impulsive agent) collected more average reward than the agent with a higher discount factor (the non-impulsive agent) by selecting immediate reward options when the probability of receiving the future reward was low. In the Information Sampling task, we manipulated the amount of information obtained with each sample. When sampling led to small information gains, the impulsive MDP agent collected more average reward than the non-impulsive agent. Third, in the Explore-Exploit task, we manipulated the substitution rate for novel options. When the substitution rate was high, the impulsive agent again performed better than the non-impulsive agent, as it explored the novel options less and instead exploited options with known reward values. The results of these analyses show that impulsivity can be advantageous in environments that are unexpectedly uncertain.

show abstract

The Neurobiology of Impulsive Decision-Making and Reinforcement Learning in Nonhuman Animals

Cited by 13 publications

References 168 publications

The Neuromodulatory Role of the Noradrenergic and Cholinergic Systems and Their Interplay in Cognitive Functions: A Focused Review

The Neuromodulatory Role of the Noradrenergic and Cholinergic Systems and Their Interplay in Cognitive Functions: A Focused Review

A method for studying reinforcement factors controlling impulsive choice for use in behavioral neuroscience

Environmental uncertainty and the advantage of impulsive choice strategies

Contact Info

Product

Resources

About