In the context of China’s New Urbanization Strategy, it is of great practical significance to study rural–urban population migration from the perspective of house purchases by rural households. In this paper, the Huai’an Rural Survey Dataset (872,414 households) was used to study the heterogeneity of rural households’ house purchases in different classes of urban destinations, and its influencing factors were analyzed with GeoDetector. The results show that the urban house purchase destinations preferred by farmers were county towns, townships, foreign cities, and metropolitan areas, indicating that in situ urbanization has become the main path of urbanization for farmers in Huai’an. Among the environmental influencing factors, the rural environment had the greatest influence on house purchases locally (in the township and county town), and this influence decreased with the outward shift of house purchase destinations. The housing environment, the settlement environment, and the population and family environment were the main environmental impact elements. The natural environment and the policy environment had little influence on the house-purchasing behavior of farmers, and the location environment was critical in exotic locations (metropolitan areas and foreign cities). Therefore, this paper argues that a higher demand for housing is growing in China’s less developed rural areas, creating a situation in which the metropolitan area is the core and the county town is the main contributor. In terms of policy improvements, it is important to pay more attention to small cities such as counties and to offer housing concessions and welfare to “new citizens” from rural areas, as well as to significantly improve the housing, earnings, and public service environment for those who prefer to stay in the countryside.