Recently, green innovation efficiency, which considers innovation and environmental factors, is gradually becoming important for the sustainable development of Chinese heavy polluting industries because of the increasing strictness in China’s environmental regulations. Previous studies ignore the impact of external environmental factors on the efficiency of green industry innovation and fail to explain the complex relationship between environmental and technical efficiency fully. Therefore, a non-radial directional distance function-data envelopment analysis (DDF-DEA) three-stage green innovation efficiency evaluation model was constructed to measure the green innovation efficiency of China’s heavy polluting industries objectively and explore the impact mechanism of external factors. Then, the aforementioned model was used to conduct an empirical test on China’s heavy polluting industries. Results indicate that the green innovation efficiency of heavy polluting industries is generally low in China, and the entire industry is in the transitional stage of “effective innovation but not green.” The uncertainty of the effect of the environmental regulation policy, the over-reliance on external technologies, and the scale diseconomies of industries, which are the key factors in improving the green innovation efficiency of China’s heavy polluting industries, have a significant negative impact on green innovation efficiency. The conclusions of this study can provide a useful reference for China and other emerging markets to formulate reasonable environmental regulations and green transition of heavy polluting industries.