Decoding human speech requires the brain to segment the incoming acoustic signal into meaningful linguistic units, ranging from syllables and words to phrases. Integrating these linguistic constituents into a coherent percept sets the root of compositional meaning and hence understanding. One important cue for segmentation in natural speech are prosodic cues, such as pauses, but their interplay with higher-level linguistic processing is still unknown. Here we dissociate the neural tracking of prosodic pauses from the segmentation of multi-word chunks using magnetoencephalography (MEG). We find that manipulating the regularity of pauses disrupts slow speech-brain tracking bilaterally in auditory areas (below 2 Hz) and in turn increases left-lateralized coherence of higher frequency auditory activity at speech onsets (around 25 - 45 Hz). Critically, we also find that multi-word chunks - defined as short, coherent bundles of inter-word dependencies - are processed through the rhythmic fluctuations of low frequency activity (below 2 Hz) bilaterally and independently of prosodic cues. Importantly, low-frequency alignment at chunk onsets increases the accuracy of an encoding model in bilateral auditory and frontal areas, while controlling for the effect of acoustics. Our findings provide novel insights into the neural basis of speech perception, demonstrating that both acoustic features (prosodic cues) and abstract processing at the multi-word timescale are underpinned independently by low-frequency electrophysiological brain activity.