Water is a crucial and fundamental resource. It is well known that agricultural cultivation, industrial production, and human daily life are not possible without water. Efficiently utilizing water resources is of great significance for achieving global Sustainable Development Goals (SDGs). In order to improve water use efficiency in various industries and promote water-saving development, China has been implementing water-saving society construction since 2002. Henan Province is the main grain-producing area in China, with wheat production accounting for a quarter of the country’s total. As the core area of “Central Plains Agricultural Valley” in Henan Province, Xinxiang City plays an important role in agricultural technology innovation and agricultural production. However, Xinxiang City is facing problems of water scarcity and pollution, which constrain the sustainability of agricultural production. Therefore, building a water-saving society can solve the current water problems faced by Xinxiang City and ensure the sustainable development of the economy and society. This study built an evaluation index system for water-saving society construction in Xinxiang, Henan Province, China. The proposed evaluation index system includes 20 evaluation indices from six aspects—integrated, agricultural water, industrial water, domestic water, water ecology and environment, and water-saving management—and then divides its development level into several stages. The Analytical Hierarchy Process (AHP) was adopted to calculate the index weight. Then, a comprehensive evaluation model for water-saving society construction in Xinxiang City was established by combining it with grey relative analysis (GRA). The results showed that the overall level of water-saving society construction in Xinxiang City is in the excellent stage, whereas water consumption per CNY 10,000 of GDP, the effective utilization coefficient of irrigation water, the reuse rate of industrial water, and the leakage rate of urban water supply network are all in the good stage. However, the urban recycled water utilization rate is still in the poor stage. These research results can effectively and reasonably reflect the development level of water-saving society construction in Xinxiang City and guide the continued implementation of water-saving society construction. At the same time, the comprehensive evaluation of water-saving society construction helps to formulate and adjust water resource management policies and measures; it also holds significant value for sustainable water management and combating water scarcity.