The alpine grassland on the Qinghai–Tibet Plateau (AGQTP) has undergone severe climate change. Although the carbon budget of AGQTP proved to have altered significantly, the spatiotemporal dynamics and the driving mechanism of the changes remain debated. This study estimated the gross primary productivity (GPP), ecosystem respiration (ER), and net ecosystem productivity (NEP) of the AGQTP, based on remote sensing models, and analyzed their spatiotemporal dynamics and their climatic driving mechanism. Our results showed that the GPP, ER, and NEP increased at rates of 0.002 Pg C yr−2, 0.013 Pg C yr−2, and 0.0007 Pg C yr−2, respectively, during 2000–2020, with substantial spatiotemporal variability. The changes in GPP were influenced by both temperature and precipitation, while NEP and ER were primarily affected by precipitation and temperature, respectively. However, the primary climatic driver of the carbon budget may have shifted from temperature to precipitation around 2010, and the impact of temperature on carbon sink was limited by local water conditions. Furthermore, we found that climate change, particularly precipitation variation, had notable legacy effects on the carbon budget of the AGQTP. Our findings highlight that the climatic impact on the carbon budget is dynamic and long-lasting, rather than static and short-lived, which should be considered in ecosystem carbon budget simulations and other related studies.