Musical rhythm, consisting of apparently abstract intervals of accented temporal events, has the remarkable ability to move our minds and bodies. Why do certain rhythms make us want to tap our feet, bop our heads or even get up and dance? And how does the brain process rhythmically complex rhythms during our experiences of music? In this chapter, we describe some common forms of rhythmic complexity in music and propose that the theory of predictive coding can explain how rhythm and rhythmic complexity are processed in the brain. We also consider how this theory may reveal why we feel so compelled by rhythmic tension in music. First, musical-theoretical and neuroscientific frameworks of rhythm are presented, in which rhythm perception is conceptualized as an interaction between what is heard ('rhythm') and the brain's anticipatory structuring of music ('the meter'). Second, three different examples of tension between rhythm and meter in music are described: syncopation, polyrhythm and groove. Third, we present the theory of predictive coding of music, which posits a hierarchical organization of brain responses reflecting fundamental, survival-related mechanisms associated with predicting future events. According to this theory, perception and learning is manifested through the brain's Bayesian minimization of the error between the input to the brain and the brain's prior expectations. Fourth, empirical studies of neural and behavioral effects of syncopation, polyrhythm and groove will be reported, and we propose how these studies can be seen as special cases of the predictive coding theory. Finally, we argue that musical rhythm exploits the brain's general principles of anticipation and propose that pleasure from musical rhythm may be a result of such anticipatory mechanisms.