Sulfate attack is one of the main concerns of cement-based materials. In this research, the magnesium sulfate attack resistivity of cement-based materials modified with nano silica (NS) and supplementary cementitious materials (SCMs) i.e., silica fume (SF), fly ash (FA) and ground granulated blast-furnace slag (GGBS) was studied up to 500 days. The effects of the initial curing time (7 and 28 days) on the mechanical property and volume stability of the cement mortars under magnesium sulfate attack were studied. The results showed that the NS has a better capability of improving the magnesium sulfate attack resistivity of the cement mortar compared with traditional supplementary materials, and a longer initial curing time contributed to the higher resistivity. The higher pozzolanic reactivity of NS led to a denser structure of the cement mortar and retarded the magnesium sulfate attack process. FA-added samples had poor resistance to magnesium sulfate attack due to the low pozzolanic reactivity. The resistance of the GGBS-added samples to magnesium sulfate attack was moderate among the samples.