There has been a significant movement in the past decades to develop alternative sustainable building material such as geopolymer cement/concrete to control CO2 emission. Industrial waste contains pozzolanic minerals that fulfil requirements to develop the sustainable material such as alumino-silicate based geopolymer. For example, industrial waste such as red mud, fly ash, GBFS/GGBS (granulated blast furnace slag/ground granulated blast furnace slag), rice husk ash (RHA), and bagasse ash consist of minerals that contribute to the manufacturing of geopolymer cement/concrete. A literature review was carried out to study the different industrial waste/by-products and their chemical composition, which is vital for producing geopolymer cement, and to discuss the mechanical properties of geopolymer cement/concrete manufactured using different industrial waste/by-products. The durability, financial benefits and sustainability aspects of geopolymer cement/concrete have been highlighted. As per the experimental results from the literature, the cited industrial waste has been successfully utilized for the synthesis of dry or wet geopolymers. The review revealed that that the use of fly ash, GBFS/GGBS and RHA in geopolymer concrete resulted high compressive strength (i.e., 50 MPa–70 MPa). For high strength (>70 MPa) achievement, most of the slag and ash-based geopolymer cement/concrete in synergy with nano processed waste have shown good mechanical properties and environmental resistant. The alkali-activated geopolymer slag, red mud and fly ash based geopolymer binders give a better durability performance compared with other industrial waste. Based on the sustainability indicators, most of the geopolymers developed using the industrial waste have a positive impact on the environment, society and economy.