Water stored deep in the soil profile is the primary bio-available reservoir for regional vegetation in the semiarid Loess Plateau of China. However, the planting of introduced shrubs over many years as part of the "Grain to Green Program (GGP)" has consistently lead to dried soil in areas with severe water scarcity. Knowledge of soil water storage (SWS) changes within deep profiles in water-deficient regions is critical for the sustainable development of vegetation restoration. Caragana korshinskii K. (CK) and Hippophae rhamnoides L. (HR) are widely planted in the Loess Plateau to control soil erosion. We selected these two shrubs for a study on variations in deep soil water (100-500 cm) and identified the main factors affecting deep soil water storage replenishment (SWSR) during their growing seasons. The results indicated that the mean SWS at 100-500 cm depth under HR was significantly higher than that under CK at both the beginning (352.74 mm for CK and 644.79 mm for HR) and end of the growing season (311.95 mm for CK and 529.05 mm for HR) (p < 0.01). In these ecosystems, SWS was only recharged below 340 cm under CK, which was due to vegetation characteristics. Under HR, however, soil water consumption exceeded recharge throughout the whole 100-500 cm profile. The SWSR at the 100-340 cm depth was mainly affected by sand content, which explained 28% of the variability of SWSR. At the 340-500 cm depth, the variability in SWSR was due to vegetation type. Therefore, expansion of the GGP should pay more attention to both soil water conditions and influencing factors, including appropriate vegetation selection and the altering of the microtopography.