Slow or sudden: Re-interpreting the learning curve for modern systems neuroscience

Moore, Sharlen; Kuchibhotla, Kishore V.

doi:10.1016/j.ibneur.2022.05.006

Cited by 13 publications

(14 citation statements)

References 72 publications

(80 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We measured behavioral learning using cue-evoked anticipatory licks before reward delivery [39][40][41] . Mice from both groups began to show cue-evoked licks in the first few days of conditioning (Fig 1C To fully compare learning rates between groups, we determined the first trial at which each individual showed evidence of learning using the cumulative sum of cue-evoked licks 3,30,[42][43][44][45] 2). Remarkably, Ext ITI mice learned in ~9 trials on average (8.8 ± 0.6), significantly less than the 94 (94 ± 7) trials needed for Typ ITI mice to learn (p<0.0001; Fig 1F).…”

Section: Temporal Scaling In Behavioral Learningmentioning

confidence: 99%

Reward timescale controls the rate of behavioral and dopaminergic learning

Burke

Jeong

et al. 2023

Preprint

View full text Add to dashboard Cite

How do we learn associations in the world (e.g., between cues and rewards)? Cue-reward associative learning is controlled in the brain by mesolimbic dopamine. It is widely believed that dopamine drives such learning by conveying a reward prediction error (RPE) in accordance with temporal difference reinforcement learning (TDRL) algorithms. TDRL implementations are trial-based: learning progresses sequentially across individual cue-outcome experiences. Accordingly, a foundational assumption, often considered a mere truism, is that the more cue-reward pairings one experiences, the more one learns this association. Here, we disprove this assumption, thereby falsifying a foundational principle of trial-based learning algorithms. Specifically, when a group of head-fixed mice received ten times fewer experiences over the same total time as another, a single experience produced as much learning as ten experiences in the other group. This quantitative scaling also holds for mesolimbic dopaminergic learning, with the increase in learning rate being so high that the group with fewer experiences exhibits dopaminergic learning in as few as four cue-reward experiences and behavioral learning in nine. An algorithm implementing reward-triggered retrospective learning explains these findings. The temporal scaling and few-shot learning observed here fundamentally changes our understanding of the neural algorithms of associative learning.

show abstract

Section: Temporal Scaling In Behavioral Learningmentioning

confidence: 99%

Reward timescale controls the rate of behavioral and dopaminergic learning

Burke

Jeong

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The acquisition of learned categorization behaviors likely involves two underlying processes—the acquisition of the knowledge of auditory categories, and the expression of this knowledge by learning the association between categories and reward outcomes ( Kuchibhotla et al, 2019 ; Moore and Kuchibhotla, 2022 ). Many models have been developed to characterize the latter process, that is, the association of a stimulus with reward.…”

Section: Resultsmentioning

confidence: 99%

Vocalization categorization behavior explained by a feature-based auditory categorization model

Kar

Pernìa

Williams

et al. 2022

eLife

View full text Add to dashboard Cite

Vocal animals produce multiple categories of calls with high between- and within-subject variability, over which listeners must generalize to accomplish call categorization. The behavioral strategies and neural mechanisms that support this ability to generalize are largely unexplored. We previously proposed a theoretical model that accomplished call categorization by detecting features of intermediate complexity that best contrasted each call category from all other categories. We further demonstrated that some neural responses in the primary auditory cortex were consistent with such a model. Here, we asked whether a feature-based model could predict call categorization behavior. We trained both the model and guinea pigs on call categorization tasks using natural calls. We then tested categorization by the model and guinea pigs using temporally and spectrally altered calls. Both the model and guinea pigs were surprisingly resilient to temporal manipulations, but sensitive to moderate frequency shifts. Critically, the model predicted about 50% of the variance in guinea pig behavior. By adopting different model training strategies and examining features that contributed to solving specific tasks, we could gain insight into possible strategies used by animals to categorize calls. Our results validate a model that uses the detection of intermediate-complexity contrastive features to accomplish call categorization.

show abstract

“…Finally, it is also worth to mention that long-term potentiation may be modulated by homeostatic plasticity 70 , 71 . Two learning processes also occur when acquiring new skills: a fast learning in a cortical structure is simultaneously slowly learned in a subcortical structure (habit learning for instance) 72 – 75 . Thus, according to previous experiences and the nature of inputs, the rate of learning may need to evolve so that neurons adapt more or less quickly to sensory inputs 69 , 72 , 76 , 77 .…”

Section: Methodsmentioning

confidence: 99%

Inhibitory neurons control the consolidation of neural assemblies via adaptation to selective stimuli

Bergoin

Torcini

Deco

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Brain circuits display modular architecture at different scales of organization. Such neural assemblies are typically associated to functional specialization but the mechanisms leading to their emergence and consolidation still remain elusive. In this paper we investigate the role of inhibition in structuring new neural assemblies driven by the entrainment to various inputs. In particular, we focus on the role of partially synchronized dynamics for the creation and maintenance of structural modules in neural circuits by considering a network of excitatory and inhibitory $$\theta$$ θ -neurons with plastic Hebbian synapses. The learning process consists of an entrainment to temporally alternating stimuli that are applied to separate regions of the network. This entrainment leads to the emergence of modular structures. Contrary to common practice in artificial neural networks—where the acquired weights are typically frozen after the learning session—we allow for synaptic adaptation even after the learning phase. We find that the presence of inhibitory neurons in the network is crucial for the emergence and the post-learning consolidation of the modular structures. Indeed networks made of purely excitatory neurons or of neurons not respecting Dale’s principle are unable to form or to maintain the modular architecture induced by the stimuli. We also demonstrate that the number of inhibitory neurons in the network is directly related to the maximal number of neural assemblies that can be consolidated, supporting the idea that inhibition has a direct impact on the memory capacity of the neural network.

show abstract

Slow or sudden: Re-interpreting the learning curve for modern systems neuroscience

Cited by 13 publications

References 72 publications

Reward timescale controls the rate of behavioral and dopaminergic learning

Reward timescale controls the rate of behavioral and dopaminergic learning

Vocalization categorization behavior explained by a feature-based auditory categorization model

Inhibitory neurons control the consolidation of neural assemblies via adaptation to selective stimuli

Contact Info

Product

Resources

About