The main problem with cement composites is that they have structural defects, including cracks, holes, and a disordered morphology, which significantly affects their strength and durability. Therefore, the construction of cement composites with defect-free structures and high strength and long durability is an important research topic. Here, by controlling the size and chemical groups of graphene oxide nanosheets (GONs) used for doping, we were able to control the entire cement matrix to form an ordered microstructure consisting of polyhedron-like crystals and exhibit flower-like patterns. The cracks and holes in the cement matrix just about vanished. The compressive and flexural strengths as well as the parameters for the durability assessment of the corresponding cement composites obviously improved compared with the control samples. Thus, the formation mechanism of the cement matrix with the ordered microstructure is proposed, and a proper explanation is given to regulation action.