Geopolymer gel that possesses advantageous features of fast setting, high strength, and good durability is increasingly used in civil engineering, including rapid retrofit projects, roadway, and other construction projects. Furthermore, geopolymer gel is also a green and economical material as it derives from solid wastes. In this study, activators with different sodium silicate modulus and alkali content were used to activate ultrafine coal gangue and slag powder to prepare coal-gangue-based geopolymers with high strength. To study the influence of slag powder content, sodium silicate modulus, and alkali activator content on strength, a two-stage design was adopted. In the first stage, the orthogonal test with three factors and four levels (10–40% slag, 0.4–1.0 modulus, 16–22%) was used to obtain the influence of each factor on the strength and select the design range of the specimen mix ratio with higher strength. In the second stage, based on the orthogonal experiment, the scope was narrowed to continue to find the optimal excitation scheme and the relationship between the influencing factors and strength. Further, mineral compositional, microstructural, functional group and elemental analyses were performed using X-ray diffraction technique, IR infrared diffraction, electron microscope observation and energy spectrum analysis to elucidate the mechanisms of the strength development. The results show that the factors affecting the geopolymer’s strength were in the order of slag content > alkali content > modulus. The optimum dosage of alkali activator was 18–20%, and the sodium silicate modulus was 0.6–0.8, and the compressive and flexural strength could reach above 40 MPa and 5.9 MPa, respectively. The compressive strength and modulus were in a parabolic relationship. Three types of cementing gels (N-A-S-H, C-A-S-H, and C-N-A-S-H) that were characterized with dense structure and high strength were identified from coal gangue and slag powder after alkali excitation.