In the Hetao Irrigation District (HID), reduction of the diverted Yellow River water for agriculture makes partial replacement of fresh water for irrigation by shallow saline groundwater a possible way to alleviate the shortage of freshwater resources. In this study, a SWAP‐WOFOST agro‐hydrological model was calibrated and validated, and then applied in a distributed manner to estimate crop yield, water productivity (WP) of spring wheat, spring maize, and sunflower, and soil salinity with conjunctive use of saline and non‐saline irrigation water during 2000–2010, and a crop zoning was proposed based on simulated crop WP in the HID. Compared with non‐saline water irrigation, the annual average yield of spring wheat, spring maize, and sunflower was changed by −0.2, 5.3, and 8.2%, and the WP was changed by −5.5, 2.6, and 7.6%, respectively, with conjunctive use of saline and non‐saline irrigation water according to the proposed crop zoning. Soil salinity in the rooting zone did not increase with time. The salts remaining in the subsoil were very little, which did not obviously affect the groundwater salinity. Our results demonstrated that partial substitution of fresh water by shallow saline groundwater for irrigation is feasible in the HID. Copyright © 2017 John Wiley & Sons, Ltd.