The difficulty of balance between environment and energy consumption makes countries and enterprises face a dilemma, and improving energy efficiency has become one of the ways to solve this dilemma. Based on the data of 158 countries from 1980 to 2018, the dynamic TFP of different countries is calculated by means of the Super-SBM-GML model. The TFP is decomposed into indexes of EC (Technical Efficiency Change), TC (Technological Change) and EC has been extended to PEC (Pure Efficiency Change) and SEC (Scale Efficiency Change). Then the fixed effect model and the fixed effect panel quantile model are used to analyze the moderating effect and the exogenous effect of energy efficiency on PM2.5 concentration on the basis of verifying that energy efficiency can reduce PM2.5 concentration. We conclude, first, the global energy efficiency has been continuously improved during the sample period, and both technological progress and technical efficiency have been improved. Second, the impact of energy efficiency on PM2.5 is heterogeneous which is reflected in the various elements of energy efficiency decomposition. The increase in energy efficiency can inhibit PM2.5 concentration and the inhibition effect mainly comes from TC and PEC, but SEC promotes PM2.5 emission. Third, energy investment plays a moderating role in the environmental protection effect of energy efficiency. Fourth, the impact of energy efficiency on PM2.5 concentration is heterogeneous in terms of national attribute, which is embodied in the differences of national development, science and technology development level, new energy utilization ratio, and the role of international energy trade.