Understanding the complex interplay between water management infrastructure and groundwater dynamics is crucial for sustainable resource utilization. This study investigates water infiltration dynamics in the secondary perched reach of the Yellow River after the operation of the Xiaolangdi Reservoir. The methodology included the application of the single-factor analysis of variance and water balance method, alongside a dual-structure, one-dimensional seepage model to simulate interactions within the system, while exploring characteristics of the groundwater flow system and the exploitation depth of below 100 m. Furthermore, we studied the influence zone range and alterations in river water infiltration in the secondary perched reach of the river following the operation of Xiaolangdi Reservoir. The results show that before the operation of the reservoir, the influence ranges of the north and south banks of the aboveground reach extended from 20.13 km to 20.48 km and 15.85 km to 16.13 km, respectively. Following the initiation of the reservoir, the river channel underwent scouring, leading to enhanced riverbed permeability. Additionally, the influence of long-term groundwater exploitation on both riverbanks extended the influence range of groundwater recharge within the secondary perched reach of the river. The influence zone of the north bank is now 23.41 km–26.74 km and the south bank 18.43 km–21.05 km. After years of shallow groundwater extraction, multiple groundwater depression cones emerged within the five major groundwater source areas on both sides of the river. Notably, deeper water levels (Zhengzhou to Kaifeng) have significantly decreased, with a drop of 42 m to 20 m to 15 m. This change in groundwater dynamics extended beyond the main channel of the river, creating a localized shallow groundwater field.