Brain mechanism of foraging: reward-dependent synaptic plasticity or neural integration of values?

Pereira-Obilinovic, Ulises; Hou, Han; Svoboda, Karel; Wang, Xiaojing

doi:10.1101/2022.09.25.509030

Cited by 4 publications

(6 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While both synaptic plasticity and non-plasticity mechanisms can explain the observed behaviors, each makes different testable assumptions about the underlying neural architecture (23) and the effect of changing environmental conditions on the behavior. For example, if one eliminated reward baiting in our experiment, a circuit using a covariance-based plasticity rule would still give rise to behavior that follows Herrnstein's matching law.…”

Section: Does the Ubiquity Of Operant Matching Imply A Common Mechani...mentioning

confidence: 99%

“…It has been hypothesized that animals that use this operant matching strategy make use of the expectation of reward -the recency-weighted rolling average over past rewards -to learn option-reward associations (18)(19)(20). Many studies further posit that this learning involves synaptic plasticity (21)(22)(23), and theoretical work has identified a characteristic relationship between operant matching and a specific form of expectation-based plasticity rule that incorporates the covariance between reward and neural activity (24)(25)(26). Despite this strong link between plasticity rules and the matching strategy, there has been no mapping of these rules onto particular synapses or plasticity mechanisms in any animal.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reward expectations direct learning and drive operant matching inDrosophila

Rajagopalan

Darshan

Fitzgerald

et al. 2022

Preprint

View full text Add to dashboard Cite

Foraging animals must use decision-making strategies that dynamically account for uncertainty in the world. To cope with this uncertainty, animals have developed strikingly convergent strategies that use information about multiple past choices and reward to learn representations of the current state of the world. However, the underlying learning rules that drive the required learning have remained unclear. Here, working in the relatively simple nervous system of Drosophila, we combine behavioral measurements, mathematical modeling, and neural circuit perturbations to show that dynamic foraging depends on a learning rule incorporating reward expectation. Using a novel olfactory dynamic foraging task, we characterize the behavioral strategies used by individual flies when faced with unpredictable rewards and show, for the first time, that they perform operant matching. We build on past theoretical work and demonstrate that this strategy requires the existence of a covariance-based learning rule in the mushroom body - a hub for learning in the fly. In particular, the behavioral consequences of optogenetic perturbation experiments suggest that this learning rule incorporates reward expectation. Our results identify a key element of the algorithm underlying dynamic foraging in flies and suggest a comprehensive mechanism that could be fundamental to these behaviors across species.

show abstract

Section: Does the Ubiquity Of Operant Matching Imply A Common Mechani...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Reward expectations direct learning and drive operant matching inDrosophila

Rajagopalan

Darshan

Fitzgerald

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Neural activity in the parietal and the frontal cortex is correlated with action values 37,39,38,55,56,40 . Updating the neural dynamics encoding action values may require vectorial error signals associated not just with reward values but also with specific actions 41,57 . Conjunctive encoding of target location and reward outcome in ALM neurons ( Fig.…”

Section: Discussionmentioning

confidence: 99%

Connectivity underlying motor cortex activity during naturalistic goal-directed behavior

Finkelstein,

Daie,

Rózsa

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Neural representations of information are shaped by local network interactions. Previous studies linking neural coding and cortical connectivity focused on stimulus selectivity in the sensory cortex1–4. Here we study neural activity in the motor cortex during naturalistic behavior in which mice gathered rewards with multidirectional tongue reaching. This behavior does not require training and thus allowed us to probe neural coding and connectivity in motor cortex before its activity is shaped by learning a specific task. Neurons typically responded during and after reaching movements and exhibited conjunctive tuning to target location and reward outcome. We used an all-optical5,4,6,7method for large-scale causal functional connectivity mappingin vivo. Mapping connectivity between > 20,000,000 excitatory neuronal pairs revealed fine-scale columnar architecture in layer 2/3 of the motor cortex. Neurons displayed local (< 100 µm) like-to-like connectivity according to target-location tuning, and inhibition over longer spatial scales. Connectivity patterns comprised a continuum, with abundant weakly connected neurons and sparse strongly connected neurons that function as network hubs. Hub neurons were weakly tuned to target-location and reward-outcome but strongly influenced neighboring neurons. This network of neurons, encoding location and outcome of movements to different motor goals, may be a general substrate for rapid learning of complex, goal-directed behaviors.

show abstract

“…While both synaptic plasticity and nonplasticity mechanisms can explain the observed behaviors, each makes different testable assumptions about the underlying neural architecture ( 18 ) and the effect of changing environmental conditions on the behavior. For example, if one eliminated reward baiting in our experiment, a circuit using a covariance-based plasticity rule would still give rise to behavior that follows Herrnstein’s matching law.…”

Section: Discussionmentioning

confidence: 99%

“…It has been hypothesized that animals that use this operant matching strategy make use of the expectation of reward—the recency-weighted rolling average over past rewards—to learn option–reward associations ( 14 , 15 ). Many studies further posit that this learning involves synaptic plasticity ( 16 – 18 ), and theoretical work has identified a characteristic relationship between operant matching and a specific form of expectation-based plasticity rule that incorporates the covariance between reward and neural activity ( 19 ). Despite this strong link between plasticity rules and the matching strategy, there has been no mapping of these rules onto particular synapses or plasticity mechanisms in any animal.…”

mentioning

confidence: 99%

Reward expectations direct learning and drive operant matching in Drosophila

Rajagopalan,

Darshan,

Hibbard

et al. 2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Foraging animals must use decision-making strategies that dynamically adapt to the changing availability of rewards in the environment. A wide diversity of animals do this by distributing their choices in proportion to the rewards received from each option, Herrnstein’s operant matching law. Theoretical work suggests an elegant mechanistic explanation for this ubiquitous behavior, as operant matching follows automatically from simple synaptic plasticity rules acting within behaviorally relevant neural circuits. However, no past work has mapped operant matching onto plasticity mechanisms in the brain, leaving the biological relevance of the theory unclear. Here, we discovered operant matching in Drosophila and showed that it requires synaptic plasticity that acts in the mushroom body and incorporates the expectation of reward. We began by developing a dynamic foraging paradigm to measure choices from individual flies as they learn to associate odor cues with probabilistic rewards. We then built a model of the fly mushroom body to explain each fly’s sequential choice behavior using a family of biologically realistic synaptic plasticity rules. As predicted by past theoretical work, we found that synaptic plasticity rules could explain fly matching behavior by incorporating stimulus expectations, reward expectations, or both. However, by optogenetically bypassing the representation of reward expectation, we abolished matching behavior and showed that the plasticity rule must specifically incorporate reward expectations. Altogether, these results reveal the first synapse-level mechanisms of operant matching and provide compelling evidence for the role of reward expectation signals in the fly brain.

show abstract

Brain mechanism of foraging: reward-dependent synaptic plasticity or neural integration of values?

Cited by 4 publications

References 63 publications

Reward expectations direct learning and drive operant matching inDrosophila

Reward expectations direct learning and drive operant matching inDrosophila

Connectivity underlying motor cortex activity during naturalistic goal-directed behavior

Reward expectations direct learning and drive operant matching in Drosophila

Contact Info

Product

Resources

About