Land use is an embodiment of human socio-economic activities and represents a bridge between these activities and natural systems. Rural residential land represents a space for rural residents to reside in and exhibits spatial characteristics that evolve over time, which is proof of rural socio-economic development. As one of the most developed regions in China, cities along the Beijing–Hangzhou Canal experienced significant changes in rural residential land use from 1990 to 2020. This paper analyses the spatial differentiation of rural residential land in 21 cities on the Grand Canal. Then, it explores the driving factors of this land using spatial grid analysis and the geographic detector model. According to the spatial differentiation characteristics and the driving factors of rural residential land, the study proposed an improved potential model for rural residential land improvement. Lastly, it proposes three different forms of rural residential land based on the results. The study found that (1) the change in rural residential land in the northern part of the Grand Canal was more volatile than that in the southern part. The change in rural residential land from 1990 to 2020 conformed to the pattern of cultivated land–rural residential land–urban construction land. (2) Based on the driving factors of rural residential land, the land is divided into one-dimensional cities, two-dimensional cities, and three-dimensional cities. Circular, linear, and scattered cities of different sizes were affected by socio-economic factors, transportation accessibility, and the natural environment, respectively. (3) Based on the potential scale of rural residential land consolidation, different types of development strategies were proposed through research, including constructing large-scale villages, relocating and reconstructing new villages, and constructing high-quality villages, respectively. Enhancing the scientific planning of rural residential land and its efficiency and tapping into the potential of land consolidation can offer the protection of agricultural land and the integration of urban and rural areas in the new era.