Research has revealed that summer fallow sowing improves the water use efficiency (WUE) and grain yield of winter wheat. However, wheat yields differ yearly with crop management. A field experiment over 8 years was established in the Loess Plateau to determine the role of precipitation and soil water storage in wheat yield formation under conservation tillage. The average WUE values were 7.8, 11.0, and 12.6 t·ha−1, while the average evapotranspiration (ET) values were 334.7, 365.5, and 410 mm when the yields were 3.0, 3.0–4.5, and over 4.5 t·ha−1, respectively. Compared to drill sowing (DS), high water consumption during early growth increased the spike number, grain number, and yield. In years of intermediate or low yields, wide-space sowing (WS) and furrow sowing (FS) improved the ET, WUE, spike number, grain number, and yield of wheat compared to (DS) drill sowing. When the wheat yield was 3.0–4.5 t·ha−1, higher soil water intake during jointing, anthesis, and anthesis–maturity increased the tiller number, 1000-grain weight, and yield, related to the use of suitable tillers. Synchronous increases in grain number per spike and 1000-grain weight were observed with increased soil water content at jointing, maturity, and anthesis, as well as consumption of soil water in the latter part during the growing season.