When listening to music, humans spontaneously perceive and synchronize movement to periodic pulses of meter. A growing body of evidence suggests that this widespread ability is related to neural processes that selectively enhance meter periodicities. However, to what extent these neural processes are affected by the attentional state of the listener remains largely unknown. Here, we recorded EEG while participants listened to auditory rhythms and detected small changes in tempo or pitch of the stimulus, or performed a visual task. The overall neural response to the auditory input decreased when participants attended the visual modality, indicating generally lower sensitivity to acoustic information. However, the selective contrast at meter periodicities did not differ across the three tasks. Moreover, this selective contrast could be trivially accounted for by biologically-plausible models of subcortical auditory processing, but only when meter periodicities were already prominent in the acoustic input. However, when meter periodicities were not prominent in the auditory input, the EEG responses could not be explained by low-level processing. This was also confirmed by early auditory responses that originate predominantly in early auditory areas and were recorded in the same EEG. The contrast at meter periodicities in these early responses was consistently smaller than in the EEG responses originating mainly from higher-level processing stages. Together, these results demonstrate that selective contrast at meter periodicities involves higher-level neural processes that may be engaged automatically, irrespective of behavioral context. This robust shaping of the neural representation of rhythm might thus contribute to spontaneous and effortless synchronization to musical meter in humans across cultures.