Thermal storage concrete is prepared by using alkali-activated ground-granulated blast furnace slag as cementitious material. To improve thermal conductivity, graphite aggregate is used to replace part of the coarse aggregate, and, furthermore, polypropylene fiber is added to improve the heat resistance performance of the concrete. The compressive strength of concrete specimens before and after heating (up to 450 °C) was tested, and, furthermore, scanning electron microscopy was used to investigate the structure alteration due to heating. Results showed that the partial replacement of coarse aggregate by graphite block could obviously improve the thermal conductivity of the thermal storage concrete. At the same time, the specimen with 30% graphite aggregate replacement still exhibited good mechanical properties. The mechanism of the high residual strength was investigated.