The response mechanism of groundwater to climate change and human activities in cities within arid and semi-arid regions, such as the Urban Planning Area of Baotou City (UPABC), northern China, is a complicated problem to understand. We analyzed the climate change relationships, including precipitation and temperature, and analyzed changes in human activities, such as groundwater consumption, and then statistically analyzed the main factors affecting groundwater depth. Furthermore, cross-wavelet and wavelet coherence methods were used to analyze the response relationship and hysteresis of groundwater depth to precipitation to better understand the groundwater depth response law. The results showed that the annual precipitation in the UPABC reduction rate was 3.3 mm/10 yr, and the annual average temperature increase rate was 0.43 °C/10 yr, from 1981 to 2017. The unconfined water decrease rate was 0.50 m/yr, and the confined water decrease rate was 0.7 m/yr. The unconfined and confined water depths were affected by precipitation and groundwater exploitation, respectively, with correlation coefficients of 0.58 and 0.57, respectively. The hysteresis of groundwater depth to precipitation was about 9–14 months. However, changes in groundwater depth, especially confined water depth, were greatly affected by groundwater exploitation. This reflected the imbalance in consumption and recharges in the UPABC, highlighting the long-term risk in areas relying on access to this resource. Therefore, arid inland zones of northern China, such as the UPABC, should pay more attention to the rational development of groundwater and strengthen the management and protection of groundwater resources.