Sex differences in learning from exploration

Abstract

Sex differences in cognitive processes could set the stage for sex-modulated vulnerability to neuropsychiatric disorders. While value-based decision making processes in particular have been proposed to be influenced by sex differences, the overall correct performance across sexes often show minimal differences. Computational tools allow us to uncover latent variables in reinforcement learning that define different decision making approaches, even in animals with similar correct performance. Here, we quantify sex differences in latent variables underlying behavior in a classic value-based decision-making task: a restless 2-armed bandit. While males and females had similar accuracy, they achieved this performance via different patterns of exploration. Males made more exploratory choices overall, largely because they appeared to get stuck in exploration once they had started. Females explored less, but learned more quickly when they did so. Together, these results suggest that sex exerts stronger influences on learning and decision making during periods of self-initiated exploration than during stable choices. These findings pinpoint the neural mechanisms of exploration as potentially conferring sex-biased vulnerability to addictions, neurodevelopmental disabilities, and other neuropsychiatric disorders.

Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise

“…Despite differences in the degree of anchoring across probabilities seen in males versus females in our choice data (Figure 3 and 4), we were unable to capture those effects in our computational model (Figure 5). It is possible that we were underpowered to detect sex differences in modeling decision noise given that our lab has observed it in the past (Chen, Knep, et al, 2020). Sex differences might also be better captured with an additional latent variable not defined within our model.…”

“…In humans, exploration in other decision making tasks correlates with the degree of delay discounting shown (Sadeghiyeh et al, 2020). We have recently identified exploration as a key driver of sex differences in other decision making tasks Chen, Knep, et al, 2020). Because discounting tasks are often used to compare groups of animals modeling neuropsychiatric risk factors or other individual differences such as sex differences (Grissom & Reyes, 2019;Orsini et al, 2016;Orsini & Setlow, 2017;van den Bos et al, 2013;Weafer & de Wit, 2014), it is imperative to identify methods that allow us to distinguish whether differences in behavior are due to value judgements putatively reflecting impulsivity or if exploration is a strong latent contributor to behavior.…”

Explore-exploit behaviors predict broad autism social phenotypes in general population

“…Previous studies have shown win-stay and lose-shift behaviors are not constant throughout the session, rather, they are elevated only during periods of exploration (19,21). Therefore, we ask whether changes in outcome-dependent behaviors in high aloofness individuals are due to changes in exploratory strategy over time.…”

“…To infer when exploration happens or how much individuals explore, we adopted a Hidden Markov Model (HMM) that models exploration and exploitation as two latent goal states underlying choices. Previous studies have found that labeling explore-exploit states using an HMM consistently and accurately modeled choice behavior across species (19,21,26). To examine how much individuals explore, we calculated the frequency of exploratory choices as labeled by the HMM.…”

“…People on the autism spectrum show stereotyped challenges in decisionmaking tasks, including difficulty in learning new reward associations and an increase in perseverative errors (9)(10)(11)(12)(13)(14). This raises the possibility that differences in flexibility in reward-guided decision-making predict other features of autism, including social communication issues, that may be related to differences in reward sensitivity (15,16) One powerful task for dissecting components of flexible decision-making as latent cognitive variables is the Multi-armed Bandit (17)(18)(19)(20)(21). In the bandit task, participants repeatedly choose between two or more choices whose value can only be learned through experience.…”