The practice of crop rotation can significantly impact carbon sequestration potential. In exploring whether crop rotation has the potential to improve soil carbon sequestration in China’s Loess Plateau, soil organic carbon (SOC), soil water content (SWC), soil bulk density (SBD), and soil pH were compared across the 0–1.0 m soil profile, under four crop rotation patterns: lentil–wheat–maize, wheat–potato–lentil, wheat–maize–potato, and wheat–flax–pea. The lentil–wheat–maize and wheat–maize–potato rotations have been practiced over the past 20 years, while the wheat–potato–lentil and wheat–flax–pea rotations were established in 1978 (~40 year rotations). The results showed that under the 20-year lentil–wheat–maize rotation, SOC was not significantly different to that of the wheat–maize–potato rotation, at 6.81 g kg−1 and 6.91 g kg−1, respectively. However, under the lentil–wheat–maize rotation, SWC (9.81%) and SBD (1.19 Mg m−3) were significantly higher, but soil pH (8.42) was significantly lower than the same metrics under wheat–maize–potato rotation (8.43% and 1.16 Mg m−3, and 8.50, respectively). For the 40-year rotations, SWC (9.19%) and soil pH (8.41) under the wheat–potato–lentil were not significantly different to that of the wheat–flax–pea (8.87%, and 8.40, respectively). SOC (6.06 g kg−1) was significantly lower, but SBD (1.18 Mg m−3) was significantly higher under the wheat–potato–lentil than the wheat–flax–pea (7.29 g kg−1, and 1.15 Mg m−3, respectively) rotations. Soil carbon sequestration for the lentil–wheat–maize and wheat–potato–lentil rotations was co-influenced by SWC, SBD, and soil pH, while for wheat–maize–potato and wheat–flax–pea rotations, it was co-influenced by SWC and soil pH. The economic value of the four studied crops is, in order: potato > maize > wheat > flax. The results of the present study suggest that the lentil–wheat–maize and maize–flax–pea rotations are the most suitable patterns to optimize simultaneous economic and ecological development of the study area.