Development of low thermal conductivity and high strength building materials is an emerging strategy to solve the heavy energy consumption of buildings. This study develops sustainable alkali activated materials (AAMs) for structural members from waste expanded polystyrene (EPS) beads and reduced graphene oxide (rGO) to simultaneously meet the thermal insulation and mechanical requirements of building energy conservation. It was found that the thermal conductivity of AAMs with 80 vol.% EPS and 0.04 wt.% rGO (E8–G4) decreased by 74% compared to the AAMs without EPS and rGO (E0). The 28-day compressive and flexural strengths of E8–G4 increased by 29.8% and 26.5% with the addition of 80 vol.% EPS and 0.04 wt.% rGO, compared to the sample with 80 vol.% EPS without rGO (E8). In terms of compressive strength, thermal conductivity, and cost, the efficiency index of E8–G4 was higher than those of other materials. A building model made from AAMs was designed using building information modeling (BIM) tools to simulate energy consumption, and 31.78% of total energy consumption (including heating and cooling) was saved in the building operation period in Harbin City, China. Hence, AAMs made of waste EPS beads and rGO can realize the structural and functional integrated application in the future.