The heavy pressure to improve CO2 emission control in industry requires the identification of key sub-sectors and the clarification of how they mitigate CO2 emissions through various actions. Focusing on 30 Chinese provincial regions, this study quantifies the contribution of each industrial sector to regional CO2 mitigation by combining the logarithmic mean Divisia index with attribution analysis and extract the key sectors of CO2 mitigation for each region. Results indicate that during 2010–2019, significant emission reduction was achieved through energy intensity (74%) in Beijing, while emission reductions were attained through industrial structure changes for Anhui (50%), Henan (45%), and Chongqing (45%). The contribution to emission reduction through energy structures is not significant. The production and supply of power and heat (PSPH) is a central factor in CO2 mitigation through all three inhibitive factors. Petroleum processing and coking (PPC) generally contributes to emission reduction through energy structures, while the smelting and pressing of ferrous metals (SPMF) through changes in industrial structures and energy intensity. PSPH and SPMF, in most regions, have not achieved the emission peak. Except in the case of coal mining and dressing (CMD), CO2 emissions in other key sectors have almost been decoupled from industrial development. CMD effectively promotes CO2 mitigation in Anhui, Henan, and Hunan, with larger contribution of PPC in Tianjin, Xinjiang, Heilongjiang, and that of smelting and pressing of nonferrous metals in Yunnan and Guangxi. The findings help to better identify key sectors across regions that can mitigate CO2 emissions, while analyzing the critical emission characteristics of these sectors, which can provide references to formulating region- and sector-specific CO2 mitigation measures for regions at different levels of development.