China is a country with the most water consumption, so it is lack of water resources. Industry has brought serious water pollution while driving economic development, which leads to the destruction of ecological environment. With the improvement of environmental awareness, many scholars have shifted their research direction to how to improve the ecological environment. Most studies consider the whole system as a “black box”, regardless of its internal structure. Therefore, a method to identify inefficiency is necessary and some suggestions for optimization are given. In this paper, a two-stage undesirable fixed-sum output data envelopment analysis (DEA) model is proposed. The industrial chemical oxygen demand (COD) emission during 2011–2015 are adjusted, and the efficiency values are calculated by heuristic search algorithm. The efficiency of 30 provinces and cities is divided into eastern, central and western regions. The model can identify the inefficient stage in industrial system, and find the source of low efficiency in the system. The analysis shows that the efficiency of eastern region is the highest, while the overall efficiency is inclined to the pollutant treatment stage. Finally, the paper puts forward some suggestions for the low efficiency areas, which can save water while ensuring economic benefits, and provide new direction for water pollution reduction and improve the ecological environment.