The Grain to Green Program in China which began in 1999 led to the conversion of 0.64 million ha of cropland to grassland on steep sloping landscapes. However, the pattern of natural vegetation succession following cropland has not been well represented in previous regional syntheses of land use change effects on soil organic carbon (SOC). A chronosequence study focusing on the vegetation succession and soil carbon stocks was conducted in the center of the Loess Plateau. The chronosequence included fields of 0, 2, 5, 8, 9, 10, 12, 15 and 25 years of self‐restoration after cropland abandonment, as well as a natural grassland reference. Plant coverage, species richness and plant biomass increased significantly with time of cropland abandonment. Over time, the species composition more nearly resembled a natural grasslands community. Cropland abandonment replenished SOC stocks by 3.6 kg C m−2 during the 25‐year self‐restoration, but the SOC accumulation was restricted to the upper soil profiles (0–60 cm). SOC accumulation rate was 88 g C m−2 y−1 in 0–30 cm and 55 g C m−2 y−1 in 30–60 cm soil depth, respectively. These carbon stocks were still significantly lower than those found in the natural grassland soil. Our results suggest that the recovery of plant communities and SOC stocks appears to be slow in this semiarid environment without revegetation effort along with appropriate field management, although the post‐agricultural soils have a high potential for carbon sequestration. Copyright © 2016 John Wiley & Sons, Ltd.