For the high demand for cement-based materials in buildings, improving the performance of cement-based materials has become the focus of relevant researchers. In recent years, nanomaterials have broad prospects in many fields such as architecture by virtue of their “lightweight, high strength, and strong solidity” characteristics. As a modifier of cement-based materials, it has also become a research hotspot. Graphene oxide (GO) is one of the most representative graphene-based nanomaterials. Because of its extremely high specific surface area and excellent physical properties, it has greatly improved the properties of cement-based materials. GO acts as an enhancer of cement composites that brings people unlimited imagination. The research progress of GO-modified cement-based materials is reviewed. The purpose is to point out the limitations of current research and provide a reference for later research. The dispersion treatment of GO and the properties of its modified cement-based materials are analyzed and summarized. In addition, the further research work that is needed and future development prospect are discussed.