A new type of cemented paste backfill (CPB) was prepared using sodium hydroxide (NaOH) as the activator, slag and silica fume (SF) as the binder, and tailings as the aggregate. The effects of proportion of replacement of 0%, 5%, 10%, 15%, and 20% silica fume on the properties of CPB were studied. The strength formation mechanism of CPB was explored through a combination of scanning electron microscopy (SEM), energy dispersive spectrometry (EDS), and Fourier transform infrared (FTIR) spectroscopy. The SEM images were analyzed by IMAGE J software, and the porosity of CPB with different silica fume contents was obtained. The results show that as the amount of silica fume increases, the unconfined compressive strength (UCS) increases first and then decreases. When the amount of silica fume was approximately 5%, CPB with a larger UCS can be obtained. When the silica fume content increased from 0% to 5%, because silica fume has good activity and small particles, more calcium silicate hydrate (C–S–H) gels and Mg-Al type layered double hydrotalcites (LDHs) were generated in CPB, which made it denser and improved its strength compared with the non-silica fume group. C–S–H gels were the main source of CPB strength. With a further increase in the amount of silica fume, thaumasite produced inside of CPB, reducing the content of C–S–H gels. Moreover, due to the expansion of thaumasite, it is easy to generate a large number of micro cracks in CPB, which weakens the strength of CPB.