Water shortages have become the major limiting factor for ecological protection and sustainable development in the Loess Plateau. Few studies have focused on the effects of different plant components on soil water and its response to precipitation at different time scales. This study conducted an observation of shrub plants with three treatments (natural condition (NC), canopy + roots after removing the litter (CR), and only roots (OR)) to monitor the dynamics of soil water during the rainy season of an extreme drought year in 2015. The results showed that the soil moisture content (SMC) and soil water storage (W) had a trend of OR > CR > NC. The response of the SMC to precipitation was gradually decreased and delayed for longer with increasing soil depth. Daily precipitation >10 mm was the threshold to trigger an SMC response below 20 cm of depth. The thresholds of precipitation to increase W were 2.09–2.54 mm at the daily scale and 29.40–32.56 mm at the monthly scale. The effect of precipitation on W and its change (∆W) also depended on the time scales. At the daily scale, precipitation only explained 1.6%, 0.9%, and 2.4% of the W variation in NC, CR, and OR, respectively. However, precipitation was more important for ∆W, making a contribution of 57.6%, 46.2%, and 56.6%, respectively, and the positive ∆W induced by precipitation happened more easily and frequently at deeper depths in OR. At the monthly scale, the contribution of precipitation to ∆W increased to 75.0%, 85.0%, and 86%, respectively. The ∆W of the whole rainy season was OR > NC > CR. Precipitation of the monthly scale displayed higher contributions to soil water than that of the daily scale. Plant components had different influences on soil water and its response to precipitation, which was strengthened by the roots, weakened by the canopy, and neutralized by the litter. Regular cutting of the canopy at the single-shrub scale may help increase water storage, which is useful for vegetation management and hydrologic regulation.