The sustainable development of agriculture is significant in protecting natural resources, protecting the ecological environment, ensuring food security, and eliminating poverty. Rural road construction promotes the flow of labor and capital between urban and rural areas, and plays a vital role in agricultural production and rural revitalization. This study aims to analyze the effect of rural road construction on the sustainable development of regional agriculture in China. We select five-dimensional indexes of population, society, economy, resources, and environment and use the entropy method to calculate the agricultural sustainable development index of each province in China. Then, we construct the spatial econometric model to explore the influence based on the panel data of 31 provinces in China from 2002 to 2018. The benchmark results show that rural road construction significantly promotes the sustainable development of agriculture; however, it has a negative impact on environmental sustainability, and the influence is lagging. The results are also heterogeneous among regions. Rural road construction has significantly promoted the sustainable development of agriculture in eastern and central areas, but has no significant impact on western regions. The reason is that the siphoning effect caused by the construction of rural roads has led to a loss of talents and capital in the western region, which harms the sustainable development of the population and resource system. This effect offsets the positive effect of the other three systems. This research has substantial policy implications for promoting rural revitalization and agricultural development.