Livestock grazing can strongly determine how grasslands function and their role in the carbon cycle. However, how ecosystem carbon exchange responds to grazing and the underlying mechanisms remain unclear. We measured ecosystem carbon fluxes to explore the changes in carbon exchange and their driving mechanisms under different grazing intensities (CK, control; HG, heavy grazing; LG, light grazing; MG, moderate grazing) based on a 16‐year long‐term grazing experimental platform in a desert steppe. We found that grazing intensity influenced aboveground biomass during the peak growing season, primarily by decreasing shrubs and semi‐shrubs and perennial forbs. Furthermore, grazing decreased net ecosystem carbon exchange by decreasing aboveground biomass, especially the functional group of shrubs and semi‐shrubs. At the same time, we found that belowground biomass and soil ammonium nitrogen were the driving factors of soil respiration in grazed systems. Our study indicates that shrubs and semi‐shrubs are important factors in regulating ecosystem carbon exchange under grazing disturbance in the desert steppe, whereas belowground biomass and soil available nitrogen are important factors regulating soil respiration under grazing disturbance in the desert steppe; this results provide deeper insights for understanding how grazing moderates the relationships between soil nutrients, plant biomass, and ecosystem CO2 exchange, which provide a theoretical basis for further grazing management.