Achievement of carbon neutrality requires lowering energy consumption in China. However, only a few studies have focused on the impact of urban mining on the reduction in energy consumption, and the impact of urban mining on reducing energy consumption remains unclear. This study explored the impact of urban mining on energy efficiency by adopting the panel data of 281 prefecture-level cities in China between 2003 and 2016 using the difference-in-difference method, and tested the setting up of urban mining pilot bases on reducing urban energy consumption per unit of gross domestic product (GDP) and channel mechanisms. The empirical findings show that urban mining pilot bases have significantly reduced energy consumption per unit of GDP by 3.67%. The instrumental variable method was used to overcome endogenous problems of the empirical results and related robustness tests. The verification results of the impact mechanism show that urban mining reduces the energy consumption per unit of GDP through three channels: by improving the degree of urban marketization, enhancing the harmony between the government and the market, and increasing the abundance of factor markets. The heterogeneity analysis shows that urban mining has a significant reduction effect on energy consumption per unit of GDP in all four regions of China; however, the energy-saving effect in the northeast is higher than that in the other regions. This study provides an empirical test and policy insights for the contribution of urban mining pilot bases in promoting China’s development of green industry and circular economy. It also offers a new path for cities in developing countries to promote energy conservation and achieve urban sustainability via urban mining.