The identification of factors that drive enterprise carbon efficiency and the assessment of current emission reduction policies from the perspective of enterprise heterogeneity are essential for designing more effective emission reduction policies that optimize the allocation of inter-enterprise resources. Using a panel of 602,470 observations of Chinese industrial enterprises over multiple years, we quantified the factors driving carbon efficiency and their contributions. We also examined the heterogeneity of these effects at the industry and regional levels. This was done by constructing multiple fixed effect models and the Shapley value decomposition model. Additionally, we evaluated carbon reduction policies related to the key driving factors, taking into account the heterogeneity of the enterprise. The study discovered significant variations in inter-enterprise carbon efficiency, with a trend of local leadership and overall trailing. Based on the results of Shapley value decomposition, industry differences, enterprise scale, and regional differences are the most critical factors affecting enterprise carbon efficiency. Based on the results of the t-test and robustness test, enterprise scale has a significant inverted “U”-shaped impact on that. The degree of impact is greater in the eastern region and related industries in China with higher levels of market development, environmental regulation, and carbon market penetration. Previous policies have overlooked the heterogeneity of carbon efficiency among enterprises, resulting in firms and regions with large production scales and high carbon efficiency shouldering a greater burden of emission reduction tasks. It is likely to encourage the flow of output away from enterprises or areas with high carbon efficiency and towards those with low carbon emissions, which hinders the improvement of overall carbon efficiency. The conclusions have valuable policy implications. These include the recommendation to reduce direct control of total emission reduction targets in carbon reduction policies. Instead, based on the inter-enterprise heterogeneity of carbon efficiency, the policies should focus on strengthening economic measures and developing specific carbon emission reduction policies at the provincial, industry, and enterprise levels.