Under the carbon emission pattern of “peak carbon and carbon neutrality”, the policy of crop rotation and fallow system (CRFS) is regarded as an important initiative to promote the green, low-carbon, and high-quality development of agriculture. Focusing on balanced panel data from 30 provinces in China from 2010 to 2021, this paper empirically examines the impact of CRFS on agricultural carbon emissions (ACEI) and its internal mechanism using a multi-temporal difference-in-differences model. The benchmark regression results show that CRFS can significantly reduce ACEI, and the results remain robust after validation by multiple methods. Mechanism results show that CRFS is able to reduce ACEI by reducing factor mismatch and promoting the level of agricultural services. Heterogeneity analysis results show that the arable land fallow rotation system is more conducive to promoting the reduction in agricultural carbon emission intensity in the main grain producing areas, main grain marketing areas, high land transfer areas, and plantation areas than in the grain production and marketing balanced areas, low land transfer, and animal husbandry areas. This study demonstrates the effectiveness of the CRFS policy implementation, provides a doctrinal basis for expanding the scope of CRFS implementation, and provides policy recommendations for relevant departments to improve the CRFS policy.