Reward foraging task and model-based analysis reveal how fruit flies learn value of available options

Abstract

Foraging animals have to evaluate, compare and select food patches in order to increase their fitness. Understanding what drives foraging decisions requires careful manipulation of the value of alternative options while monitoring animals choices. Value-based decisionmaking tasks in combination with formal learning models have provided both an experimental and theoretical framework to study foraging decisions in lab settings. While these approaches were successfully used in the past to understand what drives choices in mammals, very little work has been done on fruit flies. This is despite the fact that fruit flies have served as model organism for many complex behavioural paradigms. To fill this gap we developed a single-animal, trial-based decision making task, where freely walking flies experienced optogenetic sugar-receptor neuron stimulation. We controlled the value of available options by manipulating the probabilities of optogenetic stimulation. We show that flies integrate reward history of chosen options and forget value of unchosen options. We further discover that flies assign higher values to rewards experienced early in the behavioural session, consistent with formal reinforcement learning models. Finally, we also show that the probabilistic rewards affect walking trajectories of flies, suggesting that accumulated value is controlling the navigation vector of flies in a graded fashion. These findings establish the fruit fly as a model organism to explore the genetic and circuit basis of reward foraging decisions.

Reward expectations direct learning and drive operant matching inDrosophila

“…1). This design was inspired by earlier related assays for flies [52][53][54] , and foraging tasks in vertebrates 5,10,16,21 . In our Y-arena, a single fly begins a trial in an arm filled with clean air and can choose between two odor cues that are randomly assigned to the other two arms (see Supp.…”

“…However, most existing tasks rely on studying the place preference behavior of groups of flies. This limits the ability of the experimenter to provide reward contingent on actions of any one fly, making it hard to study behavior in response to probabilistic reward or measure choice distributions over time (but see [51][52][53][54][55] ).…”

“…Recent theoretical work has also attempted to formalize the features of the learning rule that is mediated by these plasticity mechanisms (63)(64)(65)(66)(67). Despite this progress, evidence has been mixed as to whether this learning rule makes use of reward expectations (68)(69)(70)(71)(72), and there is a dearth of understanding about how flies learn in natural environments (but see (73,74)). Studying foraging behaviors would allow us to not only clarify these gaps in the understanding of fly learning but could also provide an insightful framework for testing the neural computations underlying decision-making strategies such as matching.…”

The Panopticon—Assessing the Effect of Starvation on Prolonged Fly Activity and Place Preference

“…Flies show hunger-motivated ranging or foraging walks to find food; they also show explorative walks (or local searching behaviour) after food encounter ( Dethier, 1957 ; Bell et al, 1985 ; Bell, 1990 ; Corrales-Carvajal et al, 2016 ; Kim and Dickinson, 2017 ; Murata et al, 2017 ; Hughson et al, 2018 ; Mahishi and Huetteroth, 2019 ), and much has been achieved in identifying the circuits and dynamics involved in this behaviour ( Corfas et al, 2019 ; Lin et al, 2019 ; Moreira et al, 2019 ; Sayin et al, 2019 ; Seidenbecher et al, 2020 ; Behbahani et al, 2021 ). Most of these studies either used hunger-motivated behaviour to focus on the underlying navigational strategy of the flies, or they focussed on exploration–exploitation trade-offs under different motivational settings.…”

Long-lasting impact of starvation experience on fly activity and place preference

“…Flies show hunger-motivated ranging or foraging walks to find food; they also show explorative walks (or local searching behaviour) after food encounter (Dethier, 1957;Bell et al, 1985;Bell, 1990;Corrales-Carvajal et al, 2016;Kim and Dickinson, 2017;Murata et al, 2017;Hughson et al, 2018;Mahishi and Huetteroth, 2019), and much has been achieved in identifying the circuits and dynamics involved in this behaviour (Corfas et al, 2019;Lin et al, 2019;Moreira et al, 2019;Sayin et al, 2019;Seidenbecher et al, 2020). Most of these studies either used hunger-motivated behaviour to focus on the underlying navigational strategy of the flies, or they focused on exploration-exploitation trade-offs under different motivational settings.…”