The utilization of silicomanganese slag (SiMnS) as a precursor for alkali-activated materials (AAMs) is considered as an efficient approach for sustainable and eco-friendly large-scale resource utilization. However, sodium silicate solutions account for more than 50% of the production costs and carbon emissions of AAMs. In this study, AAM activators were prepared by silica-containing waste (acid leaching residue of boron mud, BM-AR) and NaOH as raw materials, and were successfully substituted for commercial sodium silicate-NaOH activators. Results indicated that the NaOH dosage had a great effect on the concentration and modulus of the activator. With the appropriate dosage of NaOH (NaOH: BM-AR = 0.4–0.7), suitable moduli of AAM activators can be produced at a wide range of solid/liquid ratios (L/S = 3–4.5) under mild conditions (80–100 °C). The compressive strength of the SiMnS AAM specimens prepared by this activator can reach 68.58 MPa, and its hydration products were mainly hydrated calcium silicate and amorphous silica–alumina gel, indicating the successful preparation of AAM. Calculation showed that the carbon emission of the AAMs prepared in this study was 12.4% and 37.6% of that of OPC and commercial water glass/NaOH-activated AAMs, and the cost was only 67.14% and 60.78% of them. The process achieves the use of waste materials to replace commercial activators, and is expected to be extended to a variety of AAMs raw materials and silica-containing waste. This makes it a highly promising alternative method for the production of AAMs, particularly the ‘just add water’ AAMs.