Land use/cover change is the main reason for the variation of ecosystem carbon storage. The study of the impact of land use on carbon storage has certain reference values for realizing high-quality development in the Yellow River Basin. In this paper, the InVEST model was used to simulate the variation of carbon storage in the Yellow River Basin in 2000, 2005, 2010, 2015, and 2020, and to predict the carbon storage in 2030 in combination with the CA-Markov model, as well as to discuss the impact of land use on carbon storage. The results showed that: (1) The variation trend of carbon storage for different land use types in the Yellow River Basin was different and was mainly manifested as a decrease of cultivated land and unused land, and an increase of forest land, grassland, water, and construction land. The carbon storage in the provincial key development prioritized zone, national development optimized zone, and provincial development optimized zone showed decreasing trends, while the national key development prioritized zone and national major grain producing zone presented a fluctuating downward trend. (2) The ecosystem carbon storage function weakened after 2000, and part of the carbon sink area transformed into a carbon source area. The area with low carbon storage was distributed in the west of the provincial key ecological function zone, and the area with high carbon storage was concentrated in the south and middle of national key ecological function zone and the east of the provincial key ecological function zone. (3) The carbon loss was largest in the urban expansion scenario (UES), followed by the natural development scenario (NDS) and ecological protection scenario (EPS). The carbon storage of different scenarios presented significant positive correlations with land use intensity.