The increased demand for water resources due to urban population and economic growth has worsened the urban water crisis. In order to address this issue, a policy of “developing a water-saving society” (namely, water-saving society policy) has been implemented in some Chinese cities. This study takes 285 cities at the prefecture level and above as the sample and uses the propensity score matching (PSM) method to analyze the effect of China’s urban water-saving society policy on the reduction of water consumption per CNY 10,000 gross domestic product (GDP) from 2005 to 2017. The results show that the water-saving society policy significantly (p < 0.01) reduced water consumption in the study period; however, the effects differed between cities with different water resource endowments, economic development level, and urban scale. Specifically, there was a positive water consumption reduction effect in cities in humid areas, with low economic development, or of large scale, while the effect was limited in cities in arid areas, with high economic development, or of small scale. Therefore, for areas where water resource supply is insufficient, water-saving policy should be designed and implemented suiting local conditions, and it is also necessary to explore more water sources.