Pisha sandstone (PS) is a unique geological structure in the Yellow River basin in China and is a general term for a rock interlayer composed of sandstone, sand shale, and muddy sandstone. The collapsibility of PS results in a high erosion rate and poor vegetation due to its low diagenetic potential and weak structural strength. This study showed that PS can be converted into geopolymer cement by mixing with a suitable alkali activator. PS was converted to geopolymer cement for construction to control soil erosion and conserve the soil and water in this area. Slag was used as a mineral additive to improve the performance of alkali-active PS geopolymer cement in this study. The influence of slag replacement level, NaOH dosages, and curing age on the compressive strength of alkali-activated PS was investigated. Scanning electron microscopy (SEM)/energy dispersive X-ray (EDS), thermogravimetric analyses (TG/DTG), and X-ray diffraction (XRD) were used to analyze the hydration products and microstructure of alkali-activated PS. The results showed that when the samples had 40 wt.% slag, 1.5 wt.% NaOH, and 4.0 wt.% Na2SiO3, their compressive strength could reach 82.0 MPa at 90 days. Compared with the samples with activator and without activator, the compressive strength increased by 6,664% and 9,011%, respectively. The hydration products were C-S-H gel, geopolymer gel, and calcium carbonate crystals. With 10 wt.% slag as a mineral additive, 1.5 wt.% NaOH, and 4 wt.% Na2SiO3 as an activator, the carbonation ratio of C-S-H gel was 49.3%.