Using the solid waste of granite powder (GP) to prepare green cementitious materials is of great significance for the sustainable development of the construction industry. In this study, a phosphoric acid–based geopolymer (PABG) was directly prepared from GP through a combination of phosphoric acid and calcination processes. Reaction mechanism of phosphoric‐activated GP is studied by techniques of X‐ray diffraction, thermal analysis, Fourier transform infrared spectroscopy, SE mode, and mercury intrusion porosimetry (MIP) taking calcination temperature, calcination time, and phosphoric acid as variables. The results indicated that the addition of 15 wt.% phosphoric acid and calcination at the temperature of 200°C for 90 min significantly increased compressive strength from 0.70 to 20.92 MPa. The calcination temperature was the most important factor in the synthesis of PABG using GP. The decrease of the peak value of mineral phase after calcination indicates that high temperature promotes the reaction between phosphoric acid and GP. According to the results of MIP, the gel generated by the reaction of phosphoric acid and GP refines the pore structure, which is conducive to the improvement of geopolymer properties. These research results provide a reference for the preparation of geopolymers directly using low pozzolanic active GP.