Legume green manure (LGM) is an excellent organic amendment conducive to soil quality and nutrient cycling; however, the use of LGM was once repealed in the rain-fed agriculture of northern China. The objective was to investigate the effects that planting LGM would bring and whether it would affect other fertilization regimes regarding the productivity and water and nutrient use efficiencies of succeeding crops. A short-term (2016–2019) field experiment was established with a split-plot design in the Loess Plateau of China, which included ten treatments consisting of two planting systems (main treatments)—conventional winter wheat monoculture (G0) and planting and incorporating LGM followed by winter wheat planting (G)—and five fertilization regimes (sub-treatments)—no fertilization (CK), basal fertilization with chemicals N, P and K (NPK), basal fertilization plus wheat straw return (NPK + S), basal fertilization plus farmyard manure application (NPK + M), and basal fertilization plus wheat straw return plus farmyard manure application (NPK + S + M). The results demonstrated that compared with G0, the G did not remarkably affect the total water consumption (WC) and water use efficiency (WUE) across the three trial wheat seasons. Specifically, during the third wheat season, the winter wheat yield of G increased by 7.5% more than that of G0 (p < 0.05). G primarily increased the N concentration in winter wheat and universally increased the uptake of N, P and K by 18.8%, 11.7% and 18.8%, respectively. The apparent use efficiencies (AUEs) of chemicals N, P and K under G were 88.0%, 102% and 93.2% higher than those under G0 (p < 0.05). In contrast, the wheat yields of NPK, NPK + S, NPK + M and NPK + S + M were 14.3%, 22.2%, 26.4% and 19.5%, respectively, higher than those of CK. The WC and WUE increased under NPK, NPK + S, NPK + M and NPK + S + M relative to the CK (p < 0.05). Compared with CK, the NPK, NPK + S, NPK + M and NPK + S + M primarily increased the N concentration in winter wheat and universally increased the uptake of N, P and K (p < 0.05). The AUEs of N, P and K were increased by 44.3–75.3%, 72.4–103% and 128–160%, respectively, by NPK + S, NPK + M and NPK + S + M compared with CK. In conclusion, the revival of planting LGM during the fallow period was considered an appropriate measure in the Loess Plateau and similar rain-fed regions due to its ability to improve the growth and nutrient utilization of subsequent winter wheat even in the short term, as well as the lack of negative effects exerted on other organic amendments in its effectiveness.