Researching the impact of factor flow on urban land use efficiency (ULUE) is significant in alleviating the contradiction between the accelerated promotion of urbanization and the inefficient use of urban land. This paper theoretically analyzes the impact mechanism of the flow of the three major factors of production—land, capital, and labor—on ULUE. On this basis, taking 54 cities in the urban agglomeration in the middle and lower reaches of the Yangtze River in China as the research object, the stochastic frontier model, fixed effect model, and K-means clustering method are used to empirically analyze the impact of the flow of labor, capital, and land on ULUE. The heterogeneous effect of factor flow on ULUE in different urbanization stages is also discussed. The results are as follows: (1) Labor flow and capital flow play a significant role in promoting ULUE; land flow plays the opposite role. (2) With the continuous improvement of the level of urbanization, the promotion effect of labor flow first decreases and then increases, while the promotion effect of capital flow continues to decrease, and the inhibitory effect of land flow is significant in the initial stage and expansion stage of urbanization, and the effect increases gradually, but it is not significant in the mature stage. (3) The level of urbanization has an obvious threshold effect on factor flow. There are single thresholds for labor and land flow, and there is a double threshold for capital flow. Research conclusions: we should continue to promote the market-oriented reform of factors, break the institutional shackles which restrict the free flow of factors, implement policies according to the city and the stage, and give full play to the positive impact of factor flow on ULUE.