Estimating the grassland carbon budget is critically important for ensuring that grassland resources are used sustainably. However, the spatiotemporal dynamics of the carbon budget and the response to grazing have not yet been characterized in Qinghai grasslands. Here, we estimated the gross primary productivity (GPP) and net ecosystem exchange (NEE) in Qinghai grasslands using the improved Biome-BGCMuSo model to characterize the spatiotemporal dynamics of the carbon budget and the response to grazing in this region from 1979 to 2018. The GPP of Qinghai grasslands fluctuated during the study period, with an average annual value of 118.78 gC/m2. The NEE of Qinghai grasslands fluctuated from 1979 to 2018, with an average value of −5.16 gC/m2. After 2,000, GPP increased, and NEE decreased in a fluctuating manner. There were clear regional differences in GPP and NEE. GPP was low in most areas of Qinghai, and GPP was high in eastern and southern Qinghai. The southern, southeastern, and northeastern parts of Qinghai were mainly carbon sinks, and the northwestern part of Qinghai and the region between the southeastern and northeastern parts of Qinghai were mainly carbon sources. Grazing generally decreased GPP and increased NEE in Qinghai grasslands from 1979 to 2018. There was spatial heterogeneity in the effect of grazing on GPP and NEE. Under grazing, GPP and NEE were significantly decreased mainly in eastern Qinghai, and GPP and NEE were significantly increased mainly in southern and eastern Qinghai. NEE was most affected by grazing in eastern Qinghai. The results of this study aid our understanding of the mechanism driving variation in the grassland carbon budget and provide new data that could be used to support local grassland management.