Watershed environments play an important supporting role in sustainable high-quality economic development in China, but they have been deteriorating. In order to solve environmental problems in the Poyang Lake River Basin brought about by economic development, the Jiangxi Provincial Government promulgated relevant river basin protection policies in 2015. However, after several years of this policy, the specific effects of its implementation are a matter of general concern to the government and academic circles. After years of policy implementation, the implementation effect of the watershed ecological compensation policy needs to be evaluated. Based on 4248 observations from the Jiangxi and Hunan Provinces, we adopt the difference-in-difference method to analyze the impact of the ecological compensation policy on the Poyang Lake River Basin. The empirical results show that the ecological compensation policy has a significant effect on water-quality improvement. Water quality in the upstream area is better than that in the downstream area; areas with small administrative areas have a smaller population, which in turn leads to better water quality in the river basin; and the higher the per capita GDP, the worse the water quality. Our results highlight the need for the following policy improvements: ecological priority, customizing measures to local conditions, tracing the main body, and strengthening supervision.