Distinct roles for direct and indirect pathway striatal neurons in reinforcement

Abstract

Dopamine signaling is implicated in reinforcement learning, but the neural substrates targeted by dopamine are poorly understood. Here, we bypassed dopamine signaling itself and tested how optogenetic activation of dopamine D1- or D2-receptor-expressing striatal projection neurons influenced reinforcement learning in mice. Stimulating D1-expressing neurons induced persistent reinforcement, whereas stimulating D2-expressing neurons induced transient punishment, demonstrating that activation of these circuits is sufficient to modify the probability of performing future actions.

Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive.

“…Notably, these findings persisted even in the presence of simulated DA blockade (simulated by altering the choice incentive parameter): Because learning effects of DA in the model result from activation of D1 or D2 MSNs, the learning asymmetry persists with DA blockade due to their direct stimulation. Importantly, this is not simply a null effect: the model does predict that DA blockade reduces the absolute number of actions selected, even without changing the relative preference between the actions; both of these effects accord well with the observations of Kravitz et al (2012; Figure 5). …”

“…Importantly, this is not simply a null effect: the model does predict that DA blockade reduces the absolute number of actions selected, even without changing the relative preference between the actions; both of these effects accord well with the observations of Kravitz et al (2012; Figure 5). Finally, Kravitz et al (2012) reported that learning induced by D2 MSN stimulation was less robust and rapidly extinguished relative to D1 MSN stimulation. However we show here that their full pattern of results are obtained in the model without assuming any asymmetry in the robustness of learning per se.…”

“…Finally, Kravitz et al (2012) reported that learning induced by D2 MSN stimulation was less robust and rapidly extinguished relative to D1 MSN stimulation. However we show here that their full pattern of results are obtained in the model without assuming any asymmetry in the robustness of learning per se.…”

“…Simulations were run to provide 30 min sessions, corresponding to the experimental design in Kravitz et al (2012). Effects of dopamine blockade were simulated by setting an asymmetry in choice parameters, with ϭ Ϫ0.3, such that ␤ G ϭ ␤ ‫ء‬ (1 ϩ ) and ␤ N ϭ ␤ ‫ء‬ (1 ϩ ).…”

“…Similar effects of dopamine manipulations have been observed in healthy and other populations (Cools et al, 2009;Frank, Moustafa, et al, 2007;Frank, Santamaria, Reilly, & Willcutt, 2007;Jocham et al, 2011;Pessiglione et al, 2006). This reinforcement learning theory of dopamine function can also account for other counterintuitive phenomena, such as aberrant learning in some situations (e.g., Beeler, Daw, Frazier, & Zhuang, 2010; Wiecki, Riedinger, von Ameln-Mayerhofer, Schmidt, & Frank, 2009: learned catalepsy), and provides a mechanism explaining progression of Parkinson's disease symptoms even without further dopaminergic degeneration (Beeler et al, 2012).More direct probing of the role of dopamine in specific neural circuits comes from optogenetic studies confirming a role for the D1 and D2 pathways in approach and avoidance learning (Kravitz, Tye, & Kreitzer, 2012). After a specific action was selected endogenously by a mouse, optogenetic stimulation of D1 MSNs resulted in positive reinforcement of that specific action, causing the mouse to repeat it in the future.…”