Breeding cultivars that can maintain high production and water productivity (WP) under various growing conditions would be important for mitigating freshwater shortage problems. Experiments were carried out to assess the changes in yield and WP of different cultivars by breeding and traits related to the changes using tubes with 1.05 m depth and 19.2 cm inner diameter buried in the field located in the North China Plain. Six winter wheat cultivars released from the 1970s to 2010s were assessed under three water levels for three seasons. The results indicated that yield was on average improved by 19.9% and WP by 21.5% under the three water levels for the three seasons for the cultivar released in the 2010s as compared with that released in the 1970s. The performance of the six cultivars was relatively stable across the experimental duration. The improvement in yield was mainly attributed to the maintenance of higher photosynthetic capacity during the reproductive growth stage and greater above-ground biomass accumulation. These improvements were larger under wet conditions than that under dry conditions, indicating that the yield potential was increased by cultivar renewal. Traits related to yield and WP improvements included the increased harvest index and reduced root: shoot ratio. New cultivars reduced the redundancy in root proliferation in the topsoil layer, which did not compromise the efficient utilization of soil moisture but reduced the metabolic input in root growth. Balanced above- and below-ground growth resulted in a significant improvement in root efficiency at grain yield level up to 40% from the cultivars released in the 1970s to those recently released. The results from this study indicated that the improved efficiency in both the above- and below-parts played important roles in enhancing crop production and resource use efficiency.