Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

Abstract

Although the cerebellum has been implicated in simple reward-based learning recently, the role of complex spikes (CS) and simple spikes (SS), their interaction and their relationship to complex reinforcement learning and decision making is still unclear. Here we show that in a context where a non-human primate learned to make novel visuomotor associations, classifying CS responses based on their SS properties revealed distinct cell-type specific encoding of the probability of failure after the stimulus onset and the non-human primate’s decision. In a different context, CS from the same cerebellar area also responded in a cell-type and learning independent manner to the stimulus that signaled the beginning of the trial. Both types of CS signals were independent of changes in any motor kinematics and were unlikely to instruct the concurrent SS activity through an error based mechanism, suggesting the presence of context dependent, flexible, multiple independent channels of neural encoding by CS and SS. This diversity in neural information encoding in the mid-lateral cerebellum, depending on the context and learning state, is well suited to promote exploration and acquisition of wide range of cognitive behaviors that entail flexible stimulus-action-reward relationships but not necessarily motor learning.

Dynamic organization of cerebellar climbing fiber response and synchrony in multiple functional modules reduces dimensions for reinforcement learning

“…These results demonstrated that the cerebellum reduces dimensions of activity in a large number of neurons to a much smaller number of components, each of which is driven by a synchronization scheme that conforms to a specific task. Interestingly, we also found that individual anatomical zones and even a single CF could contain signals from multiple components [53][54][55][56] .This study provided the first evidence simultaneously supporting the two major theories of cerebellar functions in a single task 18,[57][58][59][60][61][62] , and should contribute to resolution of the longstanding controversy 63 . This study also unveiled the secret of cerebellar functional architecture: learning from small samples is achieved by compartmentalization (reduced degrees of freedom) due to synchronization and dynamics; therefore, it may contribute to new-generation AI designs.…”

Mixed Selectivity in the Cerebellar Purkinje-Cell Response during Visuomotor Association Learning

“…A hallmark of the previously reported reward-based error signal was that it was unlikely to be driven by complex spike activity (Sendhilnathan et al, , 2021. Here, too, the concurrent complex spike activity was unlikely to have caused these changes in simple spike activity (Fig.…”

“…8E). This suggests that there is no obvious relationship between simple spike and complex spike activity during reward-based learning (Larry et al, 2019;Sendhilnathan et al, 2021). and that the complex spike responses did not affect the simple spike activity or the behavior through an error-based learning mechanism.…”

“…Historically, the cerebellum has been considered to be exclusively involved in the acquisition of motor skills (De Zeeuw and Ten Brinke, 2015), motor learning, and gain adaptation (Bastian and Lisberger, 2021). However, recent converging evidence (Heffley et al, 2018;Heffley and Hull, 2019;Kostadinov et al, 2019;Larry et al, 2019;Sendhilnathan et al, , 2021 strongly suggests that the cerebellum also processes reward-related information. In particular, the lateral area of Lobule VII of the cerebellar hemisphere, near Crus I and II, is a region that has extensive reciprocal connections with other reward processing areas of the brain, including PFC and basal ganglia (Tedesco et al, 2011;Koziol et al, 2014;Caligiore et al, 2017).…”

Activation of Cerebellar Lobule VI PCTH+–Med Pathway Selectively blocks METH-induced Conditioned Place Preference in mice

“…Rodent studies showed cocaine-paired olfactory cue increased activity of the granule cell layer of the cerebellar vermis (Carbo-Gas, Vazquez-Sanroman, Aguirre-Manzo, et al, 2014; Carbo-Gas, Vazquez-Sanroman, Gil-Miravet, et al, 2014), and a random cocaine-odour pairing procedure triggered higher cFos expression in the Med (Carbo-Gas, Vazquez-Sanroman, Gil-Miravet, et al, 2014). Non-human primate PCs encoded multiple independent reward-related signals during reinforcement learning (Sendhilnathan, Ipata, & Goldberg, 2021). Here, we found Med CaMKII neurons were notably excited during CPP training, but not CPP test, suggesting Med glutamatergic neurons are responsive to METH instead of METH-paired conditioned stimulus.…”