Human production activities and social development cannot be separated from the indispensable water resources. In recent years, with the increasing speed of China's economic development, the consumption of water resources has gradually become the focus of attention of the Chinese government and the public. Based on the original data of water use of 31 provinces (municipalities and autonomous regions) in China from 2014 to 2018, a fractional order accumulative gray prediction model (FGM(1,1)) is established to predict the water use of 31 provinces (municipalities and autonomous regions) in China in the short term, and the applicability of the model is verified by the mean absolute percentage error (MAPE) value. According to the MAPE value, the data obtained by the FGM(1,1) model is relatively accurate. The prediction results show that in case of little change in total water consumption, agricultural water and industrial water use show a downward trend, while domestic water use shows an upward trend. The government's water management policy for urban residents needs to be adjusted, and the publicity work to improve the water-saving awareness of urban residents needs to be improved.