This study proposes a novel idea of the use of coal gangue (CG) activation and preheated decarburized activated coal CG-based cemented paste backfill material (PCCPB) to realize green mining. PCCPB was prepared with preheated decarburized coal CG (PCG), FA, activator, low-dose cement, and water. This idea realized scale disposal and resource utilization of coal CG solid waste. Decarbonization and activation of CG crushed the material to less than 8 mm by preheated combustion technology at a combustion temperature of 900 °C and a decarbonization activation time of 4 min. The mechanism of the effect of different Na2SO4 dosages on the performance of PCCPB was investigated using comprehensive tests (including mechanical property tests, microscopic tests, and leaching toxicity tests). The results show that the uniaxial compressive strength (UCS) of C-S2, C-S3, and C-S4 can meet the requirements of backfill mining, among which the UCS of C-S3 with a curing time of 3 d and 28 d were 0.545 MPa and 4.312 MPa, respectively. Na2SO4 excites PCCPB at different curing time, and the UCS of PCCPB increases and then decreases with the increase in Na2SO4 dosage, and 3% of Na2SO4 had the best excitation effect on the late strength (28 d) of PCCPB. All groups’ (control and CS1-CS4 groups) leachate heavy metal ions met the requirements of groundwater class III standard, and PCCPB had a positive effect on the stabilization/coagulation of heavy metal ions (Mn, Zn, As, Cd, Hg, Pb, Cr, Ba, Se, Mo, and Co). Finally, the microstructure of PCCPB was analyzed using FTIR, TG/DTG, XRD, and SEM. The research is of great significance to promote the resource utilization of coal CG residual carbon and realize the sustainable consumption of coal CG activation on a large scale.