Examining the links and potential feedbacks between tectonics and climate requires understanding the processes and variables controlling erosion. At the orogen scale, tectonics and climate are thought to be linked through the influence of mountain elevation on orographic precipitation and glaciation; the only documented erosional processes capable of balancing rapid rock‐uplift rates are glacial erosion or coupled river incision and landsliding. Our 20 new 10Be derived catchment‐averaged denudation rates from the Western Southern Alps of New Zealand generally range between 0.6 and 9 mm/yr, within the same order of magnitude as fault‐throw rates, exhumation rates, and erosion rates estimated from suspended sediment yields and landslide inventories. Combining our data with previously published 10Be denudation rates, we find that the proportion of catchment area in the 1,500–2,000 m elevation window is the variable that best explains denudation rate variability and the disparity between rock‐uplift rates and denudation rates. This correlation indicates that enhanced erosion likely occurs at 1,500–2,000 m above sea level, where periglacial and paraglacial processes have been proposed to be most active. We find that these temperature‐controlled erosional processes, which are also elevation‐dependent, can play a greater role in modulating erosion during interglacials than precipitation or glaciation. Our data suggest that temperature‐controlled peri‐ and paraglacial erosion could be efficient enough to balance some of the fastest rock‐uplift rates on Earth. Hence, temperature‐controlled erosion could contribute to limiting orogen elevations and modulating the erosion rates dictated by rock‐uplift, playing an essential role in linking tectonics and climate.