Relying on innovation to drive green energy efficiency improvement has emerged as the key to achieving the Sustainable Development Goals (SDGs) under the pressure of the global economic downturn and carbon emission reduction targets. Using the multi-period difference-in-difference (DID) method and panel data of the 282 cities between 2004 and 2019, this study evaluates and examines the effects, channels, and heterogeneity of the impact of the national innovative city pilot policy (NICPP) on the green total factor energy efficiency (GTFEE). The results are as follows: (1) the baseline regression results show that the NICPP has a positive impact on GTFEE, and the conclusions are still valid after robustness tests and instrumental variables approach to solve endogeneity problems; (2) channel tests show that the NICPP can enhance GTFEE through three channels: industrial structure transformation, green technology innovation, and resource allocation improvement; (3) heterogeneity analysis shows that the NICPP is more effective in improving GTFEE in cities with rich science and education resources, newer industrial bases, and higher initial industrial concentration. This study offers encouraging empirical support and policy implications for employing innovative city policies to improve GTFEE, cities should adapt to local conditions to encourage sustainable development-oriented innovation, industrial clustering, and optimize the market-based allocation of energy.