Coal gangue (CG) is a residual product from coal mining and washing processes. The reutilization of CG to produce geopolymers is a low-carbon disposal strategy for this material. In this study, the calcined CG (CG700°C) was used as aluminosilicate precursors, and the effects of alkali activators (i.e., Na2SiO3/NaOH, NaOH concentration, and liquid–solid) on the mechanical characteristics and microstructure of CG700°C-based geopolymers were investigated. The findings indicated that the specimens with a liquid–solid ratio of 0.50 (G2.0-10-0.50) exhibited a compact microstructure and attained a compressive strength of 24.75 MPa. Moreover, increasing the Na2SiO3/NaOH mass ratio has shortened the setting times and facilitated geopolymer gel formation, resulting in a denser microstructure and improved compressive strength. The higher NaOH concentrations of alkali activators facilitated the dissolution of CG700°C particles, and the geopolymerization process was more dependent on the condensation of SiO4 and AlO4 ions, which promoted the formation of geopolymer networks. Conversely, an increase in the liquid–solid ratio from 0.50 to 0.65 had a negative impact on compressive strength enhancement, impeding the polycondensation rate. Examination through scanning electron microscopy and mercury intrusion porosimetry revealed that employing a lower Na2SiO3/NaOH mass ratio (G1.2-10-0.55), smaller NaOH concentrations (G2.0-8-0.55), and a higher liquid–solid ratio (G2.0-10-0.65) led to the presence of larger pores, resulting in decreased 28 days compressive strength values (15.87 MPa, 13.25 MPa, and 14.92 MPa, respectively), and a less compact structure. The results suggest that the performance of CG700°C-based geopolymers is significantly influenced by alkali activators.