The cerebellum has long been associated with motor control. However, its role in cognitive functions has attracted increasing attention recently. The uniformity of cerebellar internal structure seems at odds with its involvement in such diverse cognitive functions. Nonetheless, in cerebellar diseases such as ataxia, there is a comorbidity of motor and cognitive impairments, raising essential questions about how and to what extent the cerebellum participates in cognitive functions. This review begins by tracing the historical development of cerebellar research, suggesting that the diverse connections between the cerebellum and cerebral cortex, basal ganglia, and other subcortical nuclei form the basis for the cerebellum’s role in regulating cognitive functions. We then delve into its involvement in language, reward-based learning, working memory, and spatial cognition. Additionally, we summarize the changes in the cerebellum observed in Alzheimer’s disease (AD), Parkinson’s disease (PD), and ataxias and their impact on cognitive functions. By discussing the role and mechanism of the cerebellum in cognition in physiology and pathology from the aspects of structure and function, we aim to shed light on promising new therapeutic targets related to the cerebellum for cognitive impairment.