A theory of cerebellar learning as a spike-based reinforcement learning in continuous time and space

Abstract

Reinforcement learning (RL) is a machine learning algorithm that finds optimal solutions through exploration, making it applicable in scenarios where supervised learning cannot be utilized. The brain also uses RL as an adaptive system in a complex and changing world, and the basal ganglia are known to be involved. However, it remains unclear whether other brain regions also utilize RL. In this study, we focused on the cerebellum, which has recently been reconsidered as an RL machine rather than a supervised learning machine, and we implemented its spiking network model in an actor-critic manner. Our model successfully solved a simple RL task and a cerebellum-dependent motor learning task. Furthermore, the model reproduced results in a lesion study on the same motor learning task. These results provide a spike-based implementation of an RL algorithm and a fresh view on the learning principle of the cerebellum performing RL.