A large number of natural aggregates are used in the field of construction materials, resulting in the exhaustion of natural aggregates. Therefore, looking for an alternative will slow down the consumption of natural aggregates. The sintering method not only consumes a lot of energy to prepare aggregates but also produces a lot of pollutants. In this study, silico-manganese (SM) slag was dried, ground into powder, and used as raw material. Solid and liquid alkaline activator methods were used to prepare SM slag non-burning aggregate (SMNA) by the cold bonding method. The effects of grinding time, amounts of solid and liquid alkaline activators, curing temperature, and the amount of added fly ash on aggregate properties were investigated. The aggregate microstructure was characterized by XRD, SEM, and FTIR methods, and the toxic leaching analysis of aggregate was performed. The results showed that with a fixed amount of liquid activator (16.2% wt.) and solid activator (15% wt.) and fly ash (20% wt.), respectively, and curing was performed at room temperature, the aggregate properties were optimal: the bulk density of 1236.6–1476.9 kg/m3 and the water absorption lower than 4.9–5.5%. The apparent density was 1973.1–2281.6 kg/m3, and the bulk crushing strength was 24.7–27.9 MPa. The XRD, SEM, and FTIR results indicated that amorphous gel could be formed from SM under an alkaline activator, improving the aggregate strength. The results of toxic leaching showed that the aggregate prepared from SM exhibited environmentally friendly characteristics. The SMNA was obtained via the simple and low-energy consumption production process, paving the new way toward large-scale utilization of SM.