The adjustment of administrative divisions introduces a series of uncertain impacts on the social and economic development in the administrative region. Previous studies focused more on the economic effects of the adjustment of administrative divisions, while, in this paper, we also take environmental effects into consideration. The administrative division adjustment for Chaohu Lake is used as a quasi-natural experiment to explore the influence of the adjustment on pollution control. The synthetic control method is used in this study to access the effect of administrative division adjustment on the water quality indicators of Chaohu Lake and its internal mechanism. Some conclusions are as follows. First, after the administrative division adjustment, some water quality indicators, such as ammonia nitrogen, have indeed been alleviated; however, other major pollution indicators, such as chemical oxygen demand and dissolved oxygen, have deteriorated to varying degrees. Second, the results also reveal that improper development ideas, industrial excessive expansion, and the swing of economic growth and environmental goals are problems after the adjustment. Returning to the original intention of adjustment, rationalizing the Chaohu Lake management system and designing a sound and feasible accountability mechanism are fundamental measures to reduce pollution.