The environmental and economic problems caused by gangue accumulation continue to worsen. Therefore, the implementation of a cost-effective method for utilizing gangue resources is urgent. In this study, different gangue powder (GP) contents (0%, 10%, 20%, 30%, 40%, and 50%) for mechanical–thermal activation were used to modify a fly ash-based geopolymer (FAG). Further, the effect of GP was revealed by investigating the setting time, fluidity, porosity, water absorption rate, mechanical properties, drying shrinkage, and microstructure. Results showed that the addition of GP reduced the fluidity and setting time of gangue powder—fly ash-base geopolymer (GPFAG), improved density, and decreased the water absorption rate of GPFAG. Moreover, its mechanical properties gradually improved. Compared with GPFAG0 (FAG with 0% GP), the 28-d compressive and flexural strengths of GPFAG50 (FAG with 50% GP) increased by 246.4% and 136.8%, respectively. The incorporation of GP increased the drying shrinkage. The results of XRD and FTIR analyses showed that the addition of GP increased the production of amorphous silica–aluminate gels, such as N-S-A-H and C-S-A-H. Moreover, strong Si-O-T vibrational peaks appeared in the range 743–1470 cm−1, characterizing the GPFAG strength and reaction degree.