Grassland, as a key component of the carbon cycle in terrestrial ecosystems, is vital in confronting global climate change. Characterising the carbon density of grassland ecosystems in the Longzhong Loess Plateau is important for accurately assessing the contribution of grasslands to global climate change and achieving the goal of “peak carbon” and “carbon neutral”. In this study, the Longzhong Loess Plateau was used as the research object to explore changes in the plant–soil system carbon density in two grassland types by analysing the aboveground vegetation biomass carbon density, belowground vegetation biomass carbon density, 0–100 cm soil carbon density, and ecosystem carbon density of temperate steppe and temperate desert. The results showed that the vegetation biomass (standing and living, litter, and belowground biomass), soil, and ecosystem carbon densities of the temperate steppe were significantly higher than those of the temperate desert (p < 0.05). Their carbon densities were 700.51, 7612.95, and 8313.45 g·m−2, respectively. The vertical distribution of belowground biomass and soil carbon density in the temperate steppe was significantly higher than that in the temperate desert. The overall trend of belowground biomass carbon density in the temperate steppe and temperate desert showed a gradual decrease, whereas soil carbon density showed a steady increase. More than 91% and 96% of the carbon was stored in soil in the temperate steppe and temperate desert, respectively, and the belowground biomass carbon stock accounted for more than 84% of the total biomass carbon pools in both temperate steppe and temperate desert. Temperate steppe has a significant effect in improving the carbon stock of grassland ecosystems, so ecological protection and restoration of grassland should be strengthened in the future to enhance the capacity of grassland to sequester carbon and increase sinks.